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Abstract. We introduce a A-theoretic analogue of factorial Schur P- and Q- functions. 
These functions are non-homogeneous symmetric functions generalizing Ivanov's facto- 
rial Schur P- and Q- functions. We prove various combinatorial expressions for these 
functions, e.g. as a ratio of PfafHans, a sum over set- valued shifted tableaux, a sum over 
excited Young diagrams. As a geometric application, we show that these functions rep- 
resent the Schubert classes in the A-theory of torus equivariant coherent sheaves on the 
maximal isotropic Grassmannians of symplcctic and orthogonal types. This generalizes 
a corresponding result for the equivariant cohomology given by the authors. We also dis- 
cuss a cancellation property enjoyed by these functions, which we call A-supersymmctric 
property We prove that the A-theoretic P-functions form a (formal) basis of the ring 
of A-supersymmetric functions. 



1. Introduction 

In [52], Schur introduced a family of symmetric polynomials, now called Schur Q- 
functions, in order to describe the irreducible the characters of projective representations 
of symmetric groups. It plays distinguished parts not only in representation theory but 
also in combinatorics and geometry (see e.g. [7], [21], [22], [31] and reference therein). 
In [15], Ivanov introduced a multi-parameter deformation of the Q-functiong^, which we 
call the factorial P- and Q-functions, and proved various combinatorial formulas for them 
analogous to ones for the original Q-functions. In this paper, we introduce a ".fT-theoretic" 
analogue of Ivanov's functions, and study combinatorial properties of these functions. As 
an application, we show that these functions represent the structure sheaves of the Schu- 
bert varieties in the torus equivariant i^-theory of the maximal isotropic Grassmannians 
of symplectic or orthogonal types. 

Let n be a positive integer. Let A = (Ai,...,A r ) be a strict partition of length 
r < n, i.e. a sequence of positive integers such that Ai > • • • > A r . We define poly- 
nomials GP\(xi, . . . , x n \b) and GQ\{xi, . . . , x n \b), which we call K -theoretic factorial P- 
and Q-functions, depending on a parameter (3 and also on "equivariant parameters" 
b = {pi, b 2 , • • .)• Although these polynomials depend on (3, we do not indicate this depen- 
dence in the notation. These are non- homogeneous symmetric polynomials in Xi, . . . ,x n 
with coefficients in the polynomial ring W,[/3,b]. There are two important specializations. 
First letting (3 = 0, our functions become the factorial P- and Q-functions as studied in 
[13] . Secondly, let GP\{xi, . . . ,x n ) and GQ\{xi, . . . ,x n ) denote the functions obtained 
by the specialization bi = for all i. We simply call these functions the ./^-theoretic P- 
and Q-functions. Note that even this "non-equivariant" version has not appeared in the 
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literature before an announcement paper [12]. If we perform both specializations, then 
the functions become the classical P-and Q-functions. 

As the name suggests, our motivation of this work comes from geometry. In our previous 
paper [8] , [10] , [H] , the factorial P- and Q-functions of Ivanov were interpreted as the 
Schubert classes in equivariant cohomology of the maximal isotropic Grassmannians of 
symplectic or orthogonal types. These spaces are generalized flag varieties G/P of type C, 
and B or D (see §BJ). Our original aim is to extend this results to equivariant i^-theory. 
Thus the primary goal of this study is to prove that the polynomials GP\ and GQ\ 
represent the structure sheaves of Schubert varieties in the i^-theory of torus equivariant 
coherent sheaves on the maximal isotropic Grassmannians of symmetric and orthogonal 
types (Thm. 18. 31) . 

Regardless of geometric applications, the polynomials have independent interest. They 
will play some significant role in combinatorics, representation theory and invariant theory. 
Thus another purpose of this paper is to establish some fundamental properties of the 
polynomials, as Ivanov did in (15j. We obtained the following different expressions for 
GP\ and GQ\. 

• Hall-Littlewood-type formula (Def. 12. ip 

• Nimmo-type formula as ratio of Pfaffians (Prop. 12.11) 

• Combinatorial formula using shifted set-valued tableaux (Thm. 19. ip 

• Combinatorial formula using excited Young diagrams (Thm. 19.21) . 

The coincidence of these expressions is the main result about the combinatorics of the 
.fT-theoretic factorial P-and Q-functions. 

In the course of this study, a remarkable notion of a cancellation property for multi- 
variable polynomials arises. We call it the K- super symmetric property. Let /3 be a formal 
parameter. 

Definition 1.1. A polynomial f(x\, . . . , x n ) in Z[/3] [x\, . . . , x n ] is called K -super symmetric, 
if f(xi, . . . , x n ) is symmetric in variables x\, . . . , x n , and moreover, if for all 1 < i < j < n 
the rational function 

f(X\j • • • , tj • • • , Xj—\, 0t, • • • , Xfi) 

does not depend on t, where Qt — — t/(l + fit). 

The polynomials enjoying this property naturally form a ring, which we denote by 
GT n . A fundamental result about this ring is the basis theorem, which asserts that 
GP\(x\, . . . , x n )'8 form a Z[/3]-basis of the ring of i^-supersymmetric polynomials (see 
Thm 13. 11 for a precise statement). This generalizes work of Pragacz [29] in case of /3 — 0. 

Let us explain key ideas of our construction in more detail. For simplicity of notation, 
we restrict attention to case of type D (the maximal isotropic Grassmannians of even 
orthogonal type) in the introduction. Let 1Z be the localization of the polynomial ring 
Z[/3][£>i, b 2 , • • •] by the multiplicative system formed by the products of 1 + The ring 
TZ is essentially the ring of Laurent polynomials of infinitely many variables. In fact, 
when we specialize (3 = —1, 1Z is isomorphic to Z[e ±t:l , e ±t2 , . . .} where 6, is identified with 
1 — e li . Let GT denote the ring of i^-supersymmetric functions defined to be \\m.GT n . For 
each strict partition A we can construct a certain limit of GP\(xi, . . . ,x n \b), an element 
GP\(x\b), in the ring 71 <S>z[p] GT. The functions GP\(x\b) when A runs for the set ST of 
all strict partitions, form a 'basis' of the ring 1Z Cg>z[/3] GT over Z[/3]. 
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Let \l/ be the ring defined to be the subalgebra of Fun(iSP, TV) characterized by a K- 
theoretic analogue of the so-called Goresky-Kottwitz-MacPherson conditions (cf. [6], see 
§5. II for the definition). For each /x G SV, we define a sequence of elements in TZ ( §7.ip . 
The localization map is defined as follows: 

$ : TZ ® m GT— F(x\b) (SV 3 fi h- F(6 M |6) G TZ) 

given by substitution of x to 6 M for all \i G 5P. The image {^{GP\[x\fS)y\ \^sv is shown 
to be a family of Schubert classes (Def. 15.21) . It gives a distinguished 'basis' of \& as an 
TZ module (Thm. lTTlT) . Key facts in the proof of the identification of 'basis' is the divided 
difference equations (Thm. l6TT|) . and also vanishing property of GP\(x\b) (Prop. lTTL]) . This 
technique of identification is also used in combinatorial arguments in CLOl 

The localization map can be interpreted as the restriction map to torus fixed points. 
Let Q n be the maximal isotropic Grassmannian (of even orthogonal type). An algebraic 
torus T = (C x ) n+1 acts on Q n with finitely many fixed points naturally identified with the 
set SV(n) of strict partitions A such that Ai < n. The same set parametrize the Schubert 
varieties Q\ in Q n . The Grothendieck group Kt{Qu) of T-equivariant coherent sheaves of 
Q n has an A(T)-algebra structure, where R(T) is the representation ring of T; R(T) is 
identified with Z[e ±t] -, . . . , e ±4n+1 ]. As an A(T)-module, A^((? n ) has a free basis formed by 
the classes [CVjJt of the structure sheaves of the Schubert varieties. Let i : Q% Q n be 
the inclusion map. Then we have a pullback 

i* : K T {g n ) — > K T (Q T n ) = Fwx(SP(n),R(T)), 

which is known to be injective. If we denote by i* the component of i* at the T-fixed 
point corresponding to fi G SV(n), Thm. mi (combined with Thm. lHUj) can be restated 
in the following geometric form: 

Theorem 1.1. For all A, \i G SV(n) we have 

t;([o nx ] T ) = GP x (b,\b), 

where GP\(b^\b) is naturally considered to be an element in R(T) by specialization (3 = —1. 
Therefore, via the localization map, we have constructed canonical surjective maps 

n ® m gt — >• K T (g n ) 

for all positive integers n. This map sends GP\(x\b) to [OqJt if A G SV(n) and to 
zero otherwise (see Thm. 18751 for a more precise statement). One may think of the ring 
TZ ®z[/3] GT as an equivariant A'-ring of an infinite dimensional Grassmannian "£?oo" . This 
point of view will be discussed elsewhere. 

1.1. Organization. In Section[2l we define the A-theoretic factorial P-and Q-polynomials 
X\ , . . . , x n \b) and GP X { X\ , . . . , x n \b) in a Hall-Littlewood type form. These are sym- 
metric polynomials with coefficients in Z[/3, b]. We show a Nimmo type formula expressing 
these functions as a ratio of Pfaffians (this result is not used in the rest of this paper). 
This section include a preliminary result on the factorial Grothendieck polynomials ( [23] ) 
used in the next section. Section [3J establishes some basic results for A-supersymmetric 
polynomials (and functions). Within this section, we mainly consider the case of b = 0. 
We prove the basis theorem using a factorization property of GP\(xi, . . . ,x n ). We also 
discuss the span of GQ\(xi, . . . ,x n ). In Section HJ we sets up notation for Weyl groups 
and root systems. In Section [5j we introduce GKM ring \1/ in a combinatorial manner. 
We introduce a family of Schubert classes {^a}a and discuss its basic properties. These 
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elements (if exists) gives a natural basis for In Section [6] we prove the divided difference 
equation for GQ\(x\b) and GP\(x\b). In Section U\ we define the localization map which 
relate the ring spanned by GP\(x\b) (or GQ\(x\b)) and the GKM ring ty. The goal of this 
section is to identify ip\ and GP\(x\b) (or GQ\(x\b)) via the localization map. Section [8] 
explains geometric meaning of our polynomials and the ring of .fT-supersymmetric func- 
tions. In Section Owe state the results of expressing GP\{x\b) and GQ\{x\b) in terms of 
combinatorial objects; shifted set-valued tableaux and exited Young diagrams. Section 
[TOl is devoted to the proof the results in Section [9j 

1.2. Related works. For the (non-equivariant) .fT-theory of odd maximal orthogonal 
Grassmannians, Clifford, Thomas, and Yong [I] proved a Littlewood-Richardson rule 
which gives an explicit combinatorial description for the structure constants for the Schu- 
bert basis. In view of Cor. 18.11 these constants are equal to the structure constant for 
Xi, . . . , x n )'s. Note that [3] are based on a work of Buch and Ravikumar [3]. Our 
result in this paper is independent from their result. 

For the Lagrangian Grassmannians, Ghorpade and Raghavan [31] gave a detailed de- 
scription of Grobner degeneration of an open affine piece of a Schubert variety around a 
torus fixed point. Using this result, Kreiman [21], derived a combinatorial expression for 
the restriction of Schubert to fixed point set of the torus action class of the equivariant 
if-theory. Note also Raghavan and Upadhyay extend [31] to orthogonal types. 

It will be worth noticing that in [5j, the center of the affine BMW-algebra is identified 
with a subalgebra of the ring of symmetric Laurent polynomials having a cancelation 
property quite similar to the i^-supersymmetricity. 

It is a natural task to extend the i^-theoretic factorial P-and Q-functions to family of 
functions relevant for the equivariant .fT-theory of the full flag varieties of types B,C,D. 
Kirillov and the second named author [16] introduced such family of functions, the double 
Grothendieck polynomials of type B, C and D, by using Id-Coxeter algebra, a i^-theoretic 
analogue of the nil-Coxeter algebra. 

An outstanding open question is the determination of the structure constant for GP\(x\b) 
and GQ\(x\b). As for the "equivariant case", i.e. for the case of arbitrary b, we do not 
even have a conjecture, except for very restricted cases. 

In [T2], we introduced a bumping procedure for the set- valued shifted tableaux. As 
an application we proved a Pieri type rule for GQ\(xi, . . . ,x n ), which is combinatorially 
equivalent to a problem of counting the KLG-taJo\ean in the language of [3], where they 
derive the description utilizing a more geometric arguments. Bumping procedure for 
'orthogonal' type, which is relevant for GP\(x±, . . . ,x n ), is not known now. 

Acknowledgments. We thank Mark Shimozono for discussions and his continuous 
interest in the work. We also like to thank Leonardo Mihalcea and Alex Yong for useful 
comments on the draft. 

2. K-THEORETIC FACTORIAL SCHUR Q- AND P-FUNCTIONS 

We define iT-theoretic analogue of factorial P- and Q-functions. This generalize a form 
for P- and Q-functions as a particular case of the Hall-Littlewood functions with the 
parameter t equal to —1. These are symmetric polynomials in x±, . . . , x n with coefficients 
in Z[/3, b]. We give another expression for these polynomials as ratios of Pfaffians. We also 
give some supplementary discussion on type A case, i.e. on the factorial Grothendieck 
polynomials defined and studied by McNamara. 



^-THEORETIC ANALOGUE OF FACTORIAL SCHUR P- AND Q- FUNCTIONS 



5 



2.1. The polynomials GP\(x\, . . . ,x n \b) and GP\(xi, . . . ,x n \b). Let (3 be a parameter. 
Define binary operators © and by 

n x — y 

x®y := x + y + f3xy, x © y := . 

1 + 

We also define a deformation of /c-th powers of x with parameters b = (b\, b 2 , •••) by 

[x|of := (z © 6i)(x © 6 2 ) • ■ ■ (x © 6 fc ), 

and its variant by [[x|o]] fc := (x © x)(x © &i)(x © 62) ■ ■ • (x © 

Let A = (Ai, . . . , A r ) be a strictly decreasing sequence of positive integers. We call A 
a strict partition, and r the length of A. Let SV n denote the set of all strict partitions A 
such that the length r < n. We set [x\b} x = Y\ r j=i[ x \b} Xi and [[x\b}} x = H^itM^]^- 

Definition 2.1. Let A = (Ai, • • • , A r ) be a strict partition in SV n . We define functions 
on n variables x\, . . . , x n with parameters (3 and bi, 62, • • • as follows: 



(2.1) GP x ( Xl ,...,x n \b) := r J -^ ? E 

[n — r ! z — ' 



w£S„ 



[x\b} x n n 

2=1 J=t-f 1 



(2.2) G^On,...,^) := V w 

n — r ! ^— ' 

where w G 5 n acts as a permutation on the variables x±, . . . ,x n . 



n 



1=1 J=l + 1 J 



These functions are obviously symmetric in the variables Xi, . . . , x n , and are polynomials 
with coefficients in Z[/3] [61, b 2 , ■ ■ ■ , b m ] with m = Ai for GP A and m = Ai — 1 for GQ\. The 
fact that these are polynomials follows from a standard argument. See [23] Chap. Ill, 1, 
or [13], Prop. 1.1 (a) for a proof. We call these polynomials the K -theoretic factorial P- 
and Q -functions. 

Remark 2.1 (Comments on terminology). If we set (5 = and replace [x\b] k with the fcth 
falling factorial Yli=i( x — * + 1) in the expression ()2.1|) . the polynomial coincides with the 
factorial P-function, whose definition is due to A. Yu. Okounkov (see [13], [13] )• Ivanov 
[T5] developed the theory by using (x|a) fc := n^iC 2 - — a «) w hh arbitrary sequence of 
parameters a = (aj)j>i. The function was first called the generalized Q-(P-) functions 
in [T3], and then the multi-parameter Schur P-function in [JS]. In [5], we simply called 
Ivanov's function the factorial Schur Q-function. This terminology is consistent with a 
convention in type A case (e.g. 



Both the functions GP\(xi, . . . ,x n \b) and GQ\(xi, . . . ,x n \b) can be expanded in the 
following forms: 

GP x (x u x n \b) = k ~ W GP x (x u . . . , x n \b) k , 

k>\X\ 

GP x (x u . . . , x n \b) = P k ~ lX] GQx(x u . . . , x n \b) k , 

k>\\\ 

where GP\(xi, . . . , x n \b) k and GQ x (x±, . . . , x n \b) k are symmetric polynomials in x\, . . . , x n 
with coefficients in Z[fe] = Z[&i,&2, • • •], which are homogeneous of degree k when we set 
deg(xj) = deg(6i) = 1. The lowest homogeneous parts are the factorial Schur functions, 
^a( 

X\ j ... j x n 1 6) and Q x ( X\ j ... j x n 1 6) respectively. It is known that P\{xi, . . . , x n \b)'s 
(resp. Q x (xi, . . . , x n |6)'s) , A G SV n , are linearly independent over Z[6] (cf. [15]). From 
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this fact, it turns out that GP\(xx, . . . , x n |6)'s (resp. GP\(xi, . . . , x n |6)'s), A G SV n , are 
linearly independent over the ring Z [/?][&]. 

Remark 2.2. Here we adopt the notation in j9] for the factorial P-and Q-functions, with 
the identification bi = — tj. This is slightly different from the original one in [15]. See [9], 
§4.2 for the precise correspondence. 

The specialization obtained by setting (3 — — 1 is called the K -theoretic specialization, 
because the case is relevant when we apply these functions to A-theory. If we denote by 
F\(xi, . . . ,x n \b) the A-theoretic specialization of GP\(xi, . . . , x n \b), then we have 

(-/3)-^F x (-f3 Xl , -Px n \ - Ph, -(3b 2 , ...) = GP x (x x , ...,x n \b) 

and similarly for GQ\. Thus the case of arbitrary (3 is recovered from the A-theoretic 
specialization. In an application to torus equivariant A-theory in §8] the parameters bi 
are identified with 1 — e li where e li is a standard character of the torus. 

If all the deformation parameters bi (i > 1) are specialized to zero, then the A-theoretic 
factorial P- and Q- functions are simply denoted by GP\(x\, . . . , x n ) and GQ\(xi, . . . , x n ). 
These are relevant to the non-equivariant A'-theory of the isotropic Grassmannians. 

2.2. Some useful identities. Next result will be used in the proofs of Thm. RTTl 
Lemma 2.1. 

m m 
^2 g\ TT Xi ® Xi TT x i ® x j _|_ TT * ® x i _ ^ 

1 = 1 J^il J 1=1 

Proof. We prove this by induction on m. We can write the left-hand side as 

F{x m ) 

where F(x m ) is a polynomial in x m . We claim that degF(a; m ) < m. It is easy to see that 
degF(x m ) < m + 1. By letting X{ to 0Xj and t to 0t in (J2.3p for m — 1 we have 

m— 1 

J](l + /3x l ). 
i=i 

This equation implies that the coefficient of degree m + 1 of F(x m ) vanishes. Therefore it 
suffices to show the equation F(x m ) = (x m — t) Y\T=ii x rn ~ x d for m + 1 different values 
of x m . In fact, by letting x m = QXi (1 < i < m — 1) and x m = 0, 0i we can check the 
equality by using inductive hypothesis. □ 



m—1 m—1 , 

n^Cj "1 — r X{ Xj -i — r t t 



i=l 



j=l 



.1 ; 



Next result will be used in the proofs of Thm. [67T1 and also Prop. l2~TTl Analogous equa- 
tion for the case of (3 = can be found in [14], Prop. 2.4. 

Lemma 2.2. For k = 0, 1, 2 ; we have 

(oa\ \Sl i Rt \kTT Xi ® x i - /nr=i( 1 + ^if ~ nr=i( 1 + ifn is even 

t! j} t x i © x 3 \XTi=i( l + if" m o^. 
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Proof. Let E n ^ denote the equation. We prove E n ^ {k = 0, 1, 2) by induction on n. For 
n = 1,2, the verification is straightforward. Suppose n is odd and n > 3. Assume E m ^ 
hold for m < n, k = 0, 1, 2. We write the left-hand side of E n $ as 



where F( polynomial in x n . It is easy to see that &egF(x n ) < n. Using E n _\^ 

we see that the coefficient of x™ in F(x n ) is zero. Hence we have deg-F(x n ) < n — 1. 
Now in order to prove E n p we only have to show this for n distinct values of x n . Let us 
put x n = QXi (1 < i < n — 1). Then E n $ is reduced to E n _2,o- If we put x n = then 
E n Q is reduced to E n _i $. Thus we have E n . Note that i? n 2 is obtained from E n>0 by 
Xj t-j- 0Xj (1 < z < n). 
Next we prove 



which is equivalent to E n> \. Write the left-hand side as in (12.51) . Then by using E n _\^ 
we see that degF(x n ) < n — 1. Now evaluation x n = QXi (1 < % < n — 1) is reduced to 
E n -2,i, whereas the specialization x n = is reduced to E n _i^. Thus we obtained E n> \. 

In a similar way, -En+1,0 follows from E Uy i, E n> o, and E n -i s o. Then we have £^+1,2 by 
switching X{ 1— )■ 0Xj (1 < i < n). Finally follows from £ ni 2, -En,i, and E n _n. □ 

2.3. Nimmo type formula for GQ\, GP\. Let A be a skew-symmetric matrix of even 
size. We denote by Pf(^4) the Pfaffian of A. There is a formula which is an analogue of 
Schur's evaluation of Pfaffian ([33]): 

Lemma 2.3. 



Proof. By Knuth's theorem [T7], we only have to show this for m — 2, which is 
straightforward. Indeed, (12 .7p is a special case of (4.4) in [17]. □ 

Nimmo [28] derived an expression for Q\( ratio of Pfaflians. See also EU , 

Ex. III. 8.13. Next result is a generalization of Nimmo's formula for GQ\(xi, . . . ,x n \b) 
and GP\(xi, . . . , x n \b). 

Theorem 2.1. Let X be a strict partition of length r < n. Let m = r ifn — r is even, and 
m = r + l ifn — r is odd, and set X r+ i = 0. Let A = A (xi, . . . , x n ) be a skew symmetric 
nxn matrix with (i,j) entry (xi — Xj)/ (xi ®Xj), and let B\ is an n x m matrix with (i,j) 
entry [[xi\b]] Xm ~^ +1 (1 + (3xi)~ j . Let 



(2.6) 




(2.7) 




(2.8) 




which is a skew symmetric matrix of (n + m) x {n + m). Put 




Pf A (x u ..., 
Pf A ( Xl ,..., 



x n ) ifn is even 
x n , 0) ifn is odd 
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Then 

GQ x ( Xl , ...,x n \b) = f[(l + fer Pf ^-"^) , 
fJl Pt {x 1 ,...,x n ) 

The similar formula holds for GP\ when we use [x\b} k instead of[[x\b]] k . 

Our proof of this theorem goes along the same line as in [28] . 
Proposition 2.1 (cf. [28], (A10)). Under the same notation as in Thm. \2.1[ we have 

n 

7(1 + 0x j )~ m x GQ x (x lt ...,x n \b) 

j=i i<*<i<" 

= Yl s s n H x det QKd^C 1 + /^r™- 1 ) >< n Xw(t) ~ XwU \ 

i 1 ^ ■ ^ ^- %w(i) fcp X w (j> 

wES n , m m+l<K]<n W \J> 

where S n ^ m denote the set of permutations w of {1, . . . , n} satisfying w(l) < ■ ■ ■ < 
w(m), w(m + 1) < ••• < w(n). The similar formula holds for GP\(xi, ... ,x n \b) when 
we use [x\b] instead of [[x\b]]. 

Proof. We can rewrite the definition of GQ\(xi, . . . , x n \b) as follows: 

-. m m n 

(2.9) r J^ y Yl m^M^+^y- 1 ]! n Xw(i)(BXwU) . 

(n-m)!^- LJ L f± ii x ,„ _ X() 

V WGSn 1=1 1=1 .7=1+1 V ' V ' 

In fact, if n — r is even (so m = r), this is immediate by using x Qy = (x — y)(l + (3y). If 
n — r is odd and then m = r + 1, we can deduce this expression by using Lemma I2~2l (the 
case of k = and the number of variables is odd). Now we write the last factor as 

m n _ 
J~J J~J %w(i) fcu %w(j) J~J x w(i) x w(j) ^ •^wi(i) Xw(j) 

■ i ■ • i i X uu(j) , ... . ^ ^uifi) x w(j) . ffi ^ujfj) 

!=1 j=!+l v ; w ' !<«<J<« m+l<i<j<n v ' u ' 

Noting that rK=i(l + P x w(j)) m — rij=i(l + fi x j) m i which is invariant under S^, and 

l<i<j<n Zw W ~ l<i<j<n %i ~ Xj 

we deduce that (12. 9p equals 

- — - x n(i + /%r £ s s n H x n^eoi^a + few) 4 -"- 1 

(n — m ! - LJ - - LJ - x; — Xj ±J - 

(2.10) x n x ^- x ^) _ 

i i ^ ■ ^ ^ ^uifi) ffi x w(j) 

Now for each w £ S^m, we perform the summation over the subgroup G^, of S n stabilizing 
K(i),- • -,Xw( m )} and {x w(m+1) , . . . ,x w{n) }. Clearly G w = S m x S n _ m . We write a E G w 
as a"i(T2 correspondingly. Summation over o~\ creates det ([[x„,(j)|6]] Aj (l + /3x„,(j))-'~ m ~ 1 ) , 
while o~2 gives a factor (n — m)\ since the last factor in (12.101) is anti-symmetric in n — m 
variables (x w ( m+ i), . . . , x w ^}. This completes the proof. □ 

We recall the following rule for expanding a Pfaffian of a matrix with blocks of zeros. 
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Lemma 2.4 (cf. [28], (A8)). Let A be an n x n skew- symmetric matrix. Let m be a 
positive integer such that m < n and n — m is even. Let B = (bij) be an n x m matrix. 
Then 
(2.11) 

(A B \ m(m-l) «, 

_t B q )=(-!) 2 2^ sgn(w)det(S tu (i) ! ... !lu ( ni ) ; i ! ... )rn ,)Pf(A tu ( TO+ i) r .. )tl) ( n )), 

where B w (i^. ^ w ( m yi^ ^ m denotes anmxm matrix with entry b w ^j, and A/>(m+i),...,u;(n) 
denotes the submatrix of A whose indices of rows and columns are w{m + 1), . . . , w(n). 

Proof of Thm. \2.1\ Since n — m is even, we have by Lemma 12.31 

m+l<i<j<n Xw ^ ® Xw ^ ^<i<3<n-m \x w{m+i) ® X w(m+j) 

Then the right-hand side of equation of Prop. 12.11 becomes the form of ( 12. lip . By 
permuting columns of B we can eliminate the sign factor (-l)-V^ to obtain (l2^]l . □ 

2.4. Factorial Grothendieck polynomials. For a positive integer n, let V n denote the 
set of partitions of length less than or equal to n; i.e. V n is a set consisting of a sequence 
A = (Ai, . . . , A n ) of non- negative integers such that Ai > ■ ■ • > A n > 0. Let A G V n . Define 
the following symmetric function 

(2.i2) g>( », ^.m.^y^a^^-v., 

LLl<i<j<n\ x t X 3J 

Remark 2.3. McNamara [25] introduced the factorial Grothendieck polynomials in terms of 
set- valued semistandard tableaux. It can be proved that (I2.12p is identical to McNamara's 
function. In fact, the argument in ^TUl modified to type A case suitably, is applicable to 
show this fact. Note that we do not use this coincidence in this paper. For our present 
purpose, the expression (I2.12p is useful. 

Given a partition A e V n . Define the sequence b\ = (Qb\ 1+n , . . . , Qb Xi+n -i+i, ■ ■ ■ , 0&A n +i) 
We identify a partition with its Young diagram as usual (see [24]). 

Proposition 2.2 (cf. [23]). Let A, y, e V n . Then 

'0 if\<ty 
Ilii^exih+j-xr © K+n-i+i) if\ = fi' 
where A'- = |Aj > j}. 

Proof. It is convenient to write the definition of G\(x\b) in the following form: 

[ x |6]A+ft,-i 



Gxib.lb) 



T\-l<i<j<n( X i © X i, 



(2.13) G A (xi,...,x n |6) = w 

where p n -i — (n — 1, . . . , 2, 1, 0). Then the proposition is proved by straightforward cal- 
culations (cf. [13], [26]). □ 



Let 1Z be the localization of Z[/3] [bi, b 2 , . . .] by the multiplicative system formed by 
products of 1 + (3bi (i > 1). Note that G\(b^\b) belongs to 1Z. The following lemma was 
proved in [25]. As our definition of G\(x\b) is different from the one in [25], we give a 
proof here for completeness. 
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Lemma 2.5 (cf. [25]). G\(xi, . . . ,x n \b) (A G V n ) form an TZ-basis oflZ[xi, . . . ,x n ] Sn . 

Proof. Let m be a positive integer. Let V n ^ m be the set of partitions A in V n such that 
Ai < m. Let L n>m be the Z[/3][6]-span of m^{x) (/J G V n>m ), where m M (x) is the monomial 
symmetric function in variables Xi, . . . ,x n corresponding to fi (cf. [23], I, 2). Note that 
the highest possible power of each Xi in G\(x±, . . . , x n \b) is m (this fact can be seen from 
(I2.12j) ). Thus we can define the elements dx p ((3,b) G Z [/?][&] by 

(2.14) G x (x ll ... 1 x n \b)= MM 

m p (x) (A G V n 

,771 

We claim that det(dx p (P, b)) is a product of factors of the form 1 + for some z. Once 
the claim is proved, we can invert the system of linear equations (I2.14p over the ring 72 = 
'Z[f3][b][(l+f3bi)~ 1 (i > 1)] to express m\(xy s as 72-linear combination of G\(xi, . . . , x n |6)'s 
with A G V n ,m- Since each element in 72. [xi, . . . , i„f" are in 72 ®z[£][&] 7^ n ,m for some m we 
have the lemma. 

In order to prove the claim, we first show that det(<i^(0, 0)) = 1. Now we specialize 
(3 and b to then G\(x\, . . . , x n \b) becomes the classical Schur function S\(xi, . . . , x n ). It 
follows that, dx^O, 0) is equal to the Kostka number (cf. [24], I. 6) if |A| = \fj,\. Moreover, 
since S\(xi, . . . , x n ) is homogeneous polynomial of degree |A|, we have c?a^(0,0) = if 
|A| 7^ \fj,\. Therefore the matrix (d^ M (0, 0))\^ is lower triangular (with respect to a linear 
order which is a refinement of the dominance order) and all the entries of the main diagonal 
are 1. This implies det(d^ At (0, 0)) = 1. 

Now using Lemma 12.21 it can be proved that the only possible irreducible factors of 
det(dxfj,(/3,b)) are of the form 1 + fibi (see [25], Lemma 4.7 and an argument after the 
lemma). Because we have det(G?A M (0, 0)) = 1 we can conclude that det(c?A M (/9, b)) is actually 
a product of 1 + /3bi for some i. □ 

As a by-product of the proof of the preceding lemma, we have the following: 

Corollary 2.1. G\{x x , ...,x n ) (AG V n ) form a Z[(3]-basis ofZ[(3][x 1 , . . . ,x n ] Sn . 

Proof. Indeed, by letting b% = for all i, the proof of Lemma 12.51 works when we 
consider Z[/3] instead of 72. □ 

2.5. Factorization formula. 

Proposition 2.3. For every k > 1 let pk denote the partition (k, k — 1, ... , 2, 1). 

(1) For positive integer n we have 

(2.15) GP Pn _ 1 (xi,...,x n \b) = GP Pn _ 1 (x 1 ,...,x n )= (xiOxj), 

l<i<j<n 

(2.16) GQ Pn {x u ...,x n \b) = GQ Pn _ 1 (x 1 ,...,x n ) = (xi®Xj). 

l<i<j<n 

(2) For positive integer n and A G V n , we have 



(2.17) GP Pn _ 1+x (xi,...,x n \b) = GP Pn _ 1 (xi,...,x n )G\(x 1 ,...,x n \b), 

(2.18) GQ Pn+x (x u ... i x n ) = GQ Pn (xi,...,x n )Gx{x 1 ,...,x n \b). 
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Proof. We only prove the GP\ case. GQ\ case is similar. Note that the length of 
p n _i + A is n — 1 or n. In each cases, we symmetrize the following rational function 



l<t<j<n Xi 9 X 3 l<i<j<n lll<i<j< n y X i © X 3) 



Since Yli<i<j<n( x i ® x j) is symmetric, we have (see (12. 13[) ) 

(2.19) GP Pn _ 1+x (xi, . . . ,x n \b) = (z»©Xj) x G A (xi, . . . ,x„|6). 

l<i<j'<n 

In order to see GP^^i, . . . , ar n |6) = rii<i<j<n( x i © we set A = in (12.191) . □ 

3. i^-SUPERSYMMETRIC POLYNOMIALS 

In this section we prove that the i^-theoretic P-functions form a basis of the ring of 
if-supersymmetric polynomials (Thm. 13.11) . Here we mainly consider "non-equivariant" 
version GQ\(xi, . . . , x n ) and GP\(xi, . . . , x n ). 

3.1. Ji-supersymmetric polynomials. The polynomials with the .fT-supersymmetric 
property (Def. II. ip form a subring in Z[/3][xi, . . . ,x n }. We denote it by GY n . 

Proposition 3.1. GP\(xi, . . . , x n ) and GQ\(xi, . . . , x n ) are in GT n . 

This is a consequence of the next Lemma which was proved in [13] for the case of /3 = 0. 



Lemma 3.1. Let r < n, and f(x\, . . . ,x r ) be a polynomial in Z[/3][xi, ... ,x r ]. Then the 
following functions are K -supersymmetric polynomial 



Rn{x X , . . . , X n ) = ^ f( x w(l), • • • , X w (r)) \ 



Xw(i) © "^w(j) 



""" "T -"""-"Ti X w(i) © X w (j\ 

WGS n 1=1 3=1+1 w u/ 

If moreover f is divisible by x\ ■ ■ ■ x r then R n (xi, . . . , x n ) = ,^ r y R n (xi, . . . , x n ) has sta- 
bility, i.e. R n+ i(xx, . . . ,x n ,0) = R n (xx, . . . ,x n ). 

Proof. Let Xi = t, Xj = Qt, for arbitrary integers i,j such that 1 < i < j < n. We 
claim that each summand corresponding to w G S n does not depend on t. To see this, set 

F n (x x , ...,x n ) = ni=i n™=i+i l — - ' • In fact ) ^ one of hJ is in W 1 ). • • • > w(r)} then 

F n (x w (i), . . . , x w (n)) vanishes. If i,j G {w{r + 1), . . . , w(n)}, then by using identity 

x®txQt 



xQtx®t 

we see that F n (x w ri), . . . , x w ( n )) does not depend on t. Also f(x w ^, . . . , x w ^) is obviously 
independent from t. Hence the claim follows. 

Next we prove the second assertion. Let x n+ \ = 0. If n + 1 is in {w(l), . . . , w(r)}, then 
the corresponding term vanishes because f(x w (i), . . . , x w ( r )) = by the assumption. If 
n + 1 G {w(r + 1), . . . , w(n)}, then F n+1 (x w{1) , . . . , x w{n+1) ) = F n (x w/{1) , . . . , x w ^ n) ) where 
w' is a permutation obtained by multiplying the transposition (w(n+l),n + l) from the 
left to w. This implies the desired result. □ 

Theorem 3.1 (Basis theorem). The polynomials GP\(x%, . . . , x n ) (A G SV n ) form a Z[/5] 
fraszs of GY n . 
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3.2. Proof of Basis Theorem. We use the same strategy of proof as in [29], with the 
aid of the following lemma. 

Lemma 3.2. Let A be any unique factorization domain, (3,x,y be independent variables 
over A. If f((3,x,y) G A[/3,x,y] vanishes when we made the substitution x = Qy, then 
f(f3, x, y) is divisible by x (By. 

Proof. We can prove this lemma by applying the division algorithm for f((3,x,y) as a 
polynomial in x. □ 

Proof of Thm \3.1[ We first consider the case when n is even integer. We use induction 
on n. Let n = 2 and let f(xi,x 2 ) be a i^-supersymmetric polynomial. We may assume 
the constant term of / is zero so that /(0, 0) = 0. By ii'-supersymmetricity we have 
f(t,Qt) = /(0, 0) = 0. Then by Lemma |3~2| f(xi,x 2 ) is divisible by {x\ @x 2 ). Thus / 
can be written as f(x±, x 2 ) = (^i ®x 2 )g(xi, X2). Since g(x\, x 2 ) is a symmetric polynomial 
we can write g(xi,x 2 ) = J2\ e -p 2 c\G\(xi,X2), c\ G Z[/3]. By the factorization formula, we 
have 

f(x 1 ,x 2 ) = ^2 c\GP\ +pa (xi,x 2 ). 

Thus f(xi,x 2 ) is a Z[/3]-linear combination of GP^ii,^), A G SV 2 . 

If n > 4. We proceed as follows. f(x\, . . . , x n _2, 0, 0) is /3-supersymmetric. By in- 
duction, /' = f(xi, . . . , x n _ 2 , 0, 0) is a linear combination of GP\{x\, . . . , x n _ 2 )'s where 
A G SVn-2- Let 

/' = H c a GP x (x u x n _ 2 ), c A G Z\p]. 

A 

Consider the polynomial g{x) = c\ GP\(xi, . . . , x n ). Let h(x) = f(x) — g(x). We have 
h(xi, . . . , x n _2, t, Qt) = h(xi, . . . , x n _2, 0, 0) = 0. This implies that x n _i ®x n divides h(x). 
Since h{x) is symmetric, h(x) is a multiple of GP Pn l (xi, . . . ,x n ) = ni<j<j<n( x * ® x j)- 
Thus 

/ = 9 + GP Pn _ 1 (xi, . . . ,x n )s(x) 
where s(x) G Z[/3][xi, . . . ,x n } Sn . Write by Cor. [2TT1 

s(x) = ^ c 'xGx(xi,...,x n ), c' A GZ[/3]. 

AG-P(n) 

Hence we have 

(3.1) f(x) = ^2 c x GPx(x 1 ,...,x n )+^2c' x GPp n _ 1+ x(x 1 ,...,x n ). 

Note that if A G V n then p n -i + A G SV n . This completes the proof. 

Next consider when n is odd. Let n — 1. This case is obvious since GTi = Z[/3][a?i] by 
definition and GP m (x\) = x™ . If n > 3 the proof is the same as the even case. □ 

3.3. Characterization of subring spanned by GQ\(x). Let GT n + be the set consist- 
ing of F G GY n such that F(t, x 2 , ■ ■ ■ , x n ) —F(0, x 2 , ■ ■ ■ , x n ) is divisible by t©t. Obviously 
GT nj+ is a subring of GY n . 

Proposition 3.2 (Basis theorem for GT nj+ ). GT nt+ = © Ae5 -p n %[l3]GQ\(xi, . . . ,x n ). 
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Proof. We know that GQ\(x\, . . . ,x n ) is i^-supersymmetric. By definition, one see 
that GQ\(t, x 2 ,..., x n ) - GQ\(0, x 2 , . . . , x n ) is divisible by t ffi t. So GQ\( X\ , . . . , x n ) is 
an element of GT nj+ . Let F G GF n + . We will show that F is a Z[/3]-linear combination 
of GQ\(xi, . . . , x n ) (A G iSPn) by induction on n. We can proceed in the same way as in 
the proof for Thm. IXT] with using GQx(x±, . . . , x n ) instead of GP\(xi, . . . , x n ). □ 

3.4. Inverse limit. As with the other types of symmetric functions, our polynomials 
have the following stability property. 

Proposition 3.3. Let X be a strict partition of length r in SV n . Then 

( 1) GQx(x!, X n -!, 0) = GQ x (xi, X n -t), 

(2) GP x (x u . . . , x n _i, 0) = GPx(x h . . . , x n _i), 
where the right-hand side are zero if r = n. 

Proof. We can apply Lemma [3.11 □ 

Remark 3.1. Note that i^-supersymmetricity implies 

GP x (xx, x n _ 2 , 0, 0|6) = GP x (xx, x n - 2 \b). 

However, the property GPx(xi, . . . , 0|6) = GPx(xi, . . . , x n -i\b) does not hold in 
general; for example, we have GP\(x\, x 2 ) — X\ © x 2 , whereas GP\{x\) — X\ © b\. In 
contrast to this fact, we have GQx(x±, . . . , x n -i, 0\b) = GQx(x±, . . . , x n _i\b), which holds 
since [[x|6]] A is divisible by x± ■ ■ -x r , whereas [x\b] x is not (see Lemma [3. ip . 

Let <p n+ i : GT n+1 — > GY n be the morphism of the Z[/3] -algebras given by the specializa- 
tion x n+ i = 0. Then {GT n , (p n } form an inverse system. Let GT denote the inverse limit 
hmGT n . We call this the ring of K- super symmetric functions. Then, by the stability 
property of GPx(xi, . . . ,x n ) (Prop. 13.31) we have GPx(x) := lim GPxjxi, . . . ,x n ) G GT. 
Similarly we define GT + = bjnGT„ j+ , which is a subring of GT. Then GQx{x) : = 
hmGQ A (^i, • • • ,x n ) in Gr + . 

Remark 3.2. Both the functions GPx(x) and GQx(x) can be expressed in the following 
form: 

(3.2) F w (x) + PF W+I (x) + P 2 F lM+2 (x) + • • • , F k (x) G A fe , 

where A k (k > 0) is the space of homogeneous symmetric functions of degree k (see |24] ) . 
The initial terms F\x\(x) are Px{x) and Qx{x) respectively. In particular we see that the 
functions GPx(x) (resp. GPx(x)) , A G SV, are linearly independent over Z[/3]. 

Remark 3.3. From combinatorial results proved in §H1 one can see that all the homoge- 
neous parts Fk(x) are actually non-zero. Moreover, each Fk(x) is a non-negative linear 
combination of monomial symmetric functions m u (x) such that = k. 

Here is the 'basis' theorem for Gr. 

Proposition 3.4. Any f(x) G Gr can be expressed uniquely as a possibly infinite Z[/3]- 
linear combination of GPx 's, 

(3.3) f(x) = ca • GP x (x), c A G Z[/3], 

xesv 

such that for all positive integer n the set {A G SV n \ cx ^ 0} is finite. 
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Proof. This is a direct consequence of Thm. 13.11 □ 

We can introduce £ Z[/3] by the following expansion with possibly infinitely many 
terms: 

GP x (x)-GP ll (x) = J2^GP,(x). 

V 

We expect that the right-hand side is actually a finite sum. For type A case, the corre- 
sponding statement is true as a consequence of the explicit description of LR-coefficients 
given there ([2], Cor. 5.5). 

Conjecture 3.1. ® Ag5 p 7*[(3]GP\(x) is a subring of GT. 

Remark 3.4. Our result (Thm. 18.11) shows that the constants c A , when specialized to = 
— 1, are equal to i^-theory Littlewood- Richardson coefficients of the maximal orthogonal 
Grassmannians. Clifford, Thomas and Yong [I] has given a combinatorial interpretation 
for the LR-coefficients. We do not know the description in [4] implies Conjecture 13.11 



The algebra GT has a natural decreasing filtration GT = F° D F 1 D • • • D F k D • • • , 
defined by 



F k GY 



i c a ■ GP ^) e GT 



c A = if |A| < k 



By proposition 13.41 and a consideration of degree in Xj, one sees that _F fc is actually an 
ideal and F k ■ F l C F fc+i . The associated graded ring gr GT := @^L is naturally 

isomorphic to Z[/3] (g>z where T denote the ring of Schur P-functions, the Z-span of the 
P\(x) (A G SV). This fact follows from Remark | 



We have similar result for GT + . This is a consequence of Prop. 13.21 

Proposition 3.5. Any f(x) £ GT+ can &e expressed uniquely as a possibly infinite Z[/3]- 
linear combination of GQ\ 's, 

(3.4) /(x)=^c r GQ A (x), c A £Z[/3], 

xesv 

such that for all positive integer n the set {A £ SV n \ c\ ^ 0} is finite. 
Conjecture 3.2. © Ag5 -p %[/3]GQ\(x) is a subring of GT + . 

Remark 3.5. The Pieri type rule in [12] implies that GQ\{x)GQk{x) is a finite Z[/3] linear 
combination of GQ^xYs. 

4. Preliminary on Weyl groups and root systems 

4.1. Weyl groups. Let X = B,C,D and W = W(X OQ ) be the Weyl group of type X^. 
This is a Coxeter group whose generators {sj} ig j- are indexed by I = {0,1,2,...} for 
Boo, Coo and / = {1, 1, 2, . . .} for Doo- If W is type Coo (Boo) then the defining relations 
are s? = 1 (i — 0, 1, 2 . . .) and 

(4.1) s^i+iSi = s i+1 SiS i+ i (i > 1), 

(4.2) SiSj = SjSi (i,j>l,\i-j\>2), 

(4.3) S S!S Si = SiSqSiSq, S Si = SiS (i > 1). 
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If W is type then the defining relations are s? = 1 (i — 1, 1, 2 . . .), (14. ip . (14.21) . and 

(4.4) sis 2 si = s 2 S|S 2 , s^Sj = SjSi (j ^ 2). 
The corresponding graphs are given as follows: 

Coo (-^oo) ^ g Q Q Q Q Q . ■ 

so si S2 ■■■ S„-l s„ 

Si 

Let J n = {0, 1, . . . , n - 1} for X = 5, C and {i, 1, 2, . . . , n - 1} for X = D. Let W(X n ) 
denote the subgroups of W^-X^) generated by Sj (i G I n ). 

4.2. Weyl groups and signed permutations. Let N be the set of positive integers 
{1, 2, . . .}. Denote by N a 'negative' copy N = {1, 2, . . .} of N. A signed permutation w of 
N, by definition, is an bijection on the set NUN such that w(i) = w(i) for all z G N and 
w(i) = i for i > n for some n > 1. Denote by Soo the group of all signed permutations. 
We often denote w G Soo by one-line form w(l)w(2) ■ ■ ■ w(n) ■ ■ ■ . Note that we only have 
to specify w(i) for positive z. For example w = 52134 • • • is a signed permutation, where 
dots mean the part that w(i) = i holds. 

Define the signed permutations Sj (i > 0) by 

(4.5) s (l)=T, s (i)=i(ien, z^l), 

(4.6) Si (i)=i + 1, Si{i + l)=i, Si (j) = j (j G N, i). 

Then (Q)l . ( tOl . and ((43]) hold, and W(Coo) = V7(B«>) can be identified with ~3 00 . Let 
sj = s sis . Explicitly, we have 

(4.7) Si(l) = 2, Si (2)=T, Si (i) = i(ieN,i/l,2). 

Then W(D CX> ) can be identified with the subgroup of Soo generated by sj, Sj (i > 1), 
which we denote this subgroup by Soo,+- 

4.3. Strict partitions and Grassmannian elements. Let SV denote the set of all 

strict partitions, i.e. SV = {J n>0 SV n . For each w G Soo, define X(w) = N D w;(N). 
For example if w = 52134- ■ ■ then X(w) = {5,3,1}. Such X(w) can be considered as a 
strict partition; for example the last X(w) corresponds to (5,3,1) G SV. Thus we have 
a surjective map A : — > SV,w H- X(w). This map gives a bijection Soo/Soo = SV, 
where Soo is the subgroup of Soo consisting of 'ordinary' permutations, i.e. those w G 5*00 
such that w(N) C N. 

An element w G is called Grassmannian, if w(l) < w(2) < ■ ■ • , where the elements 
are ordered as---<2<l<l<2<---. For example w = 53124 ■ • • is Grassmannian. 
Let denote the set of Grassmanian elements in Soo- Note that S^ is equal to the set 
of elements w G Soo such that £(wsi) > £(w) for all z other than i — 0, where t is the 
length function of W{B^) = W(Coo) ^ 

Next fact is well-known. See for example [I], [10] (note that our convention here is 
different from there) . 

Proposition 4.1. The set S^ forms a set of coset representatives for Soo/Soo. Let X 

denote the resulting bijection S^ = SV. Then forw,v G S^ we have \X(w)\ = £(w), and 
X(w) C X(v) -<=>■ w < v (Bruhat-Chevalley order). 




16 



TAKESHI IKEDA AND HIROSHI NARUSE 



Next consider type D case, i.e. S r c »,+- The group £00,+ consists of the signed permu- 
tations such that the cardinality of \(w) is even. The image A(S' 00j+ ) C SV is a subset 
of A having even length. This image can be also identified with SV by "removing all the 
diagonal boxes" as illustrated by the following: 



1 x 














X 














X 














X 





Let A + : 5*00,+ — > SV denote the obtained map. For example if w = 643125 • • • then 
\+(w) = (5, 3, 2) G SV. Thus A + gives a bijection 5' 00j+ /S' 00 = SV. The set of Grassman- 



nian elements in Soo !+ is denoted by + . As in the previous case of type B, C, the set 

+ is equal to the set of elements w G Soo t+ such that £(wsi) > £{w) for all i G I — {!}. 
The corresponding result (also well-known) for type D is the following: 



-1 

'oo,+ 



Proposition 4.2. The set S ( 

denote the resulting bijection + = SV. Then for w, v G x we have |A. 
and A(iu) C A(w) w < v (Bruhat-Chevalley order). 



forms a set of coset representatives for S c 

— i 

00,' 



-/(S'o 



Let A_ 



[w 



Note that the length function £ and Bruhat order are those of type D c 



Through the bijections given in Prop. 14.11 and Prop. 14.21 Weyl group acts naturally on 
SV. We will describe the action explicitly. Define c(a) G / of a G D(A) to be j — i for 
type B and C case. For type D case, we define c(i,j) = j — i + 1 (z < j) and c(z, z) = 1 
of i is even and c(i,i) = 1 of z is odd. For each type, a strict partition A is i-removable if 
there is a box a G D(A) with c{a) = i such that D(A) — {«} is a shifted diagram. Then 
we denote the diagram D(A) — {a} by AW. Conversely A is i-addable if there is /1 such 
that n is z-removable and fj,w = A. 

Proposition 4.3. Let A G SV. Then 

• SjA < A <^=^ A is i-removable, 

• SjA > A <^=^> A is i-addable, 

• SjA = A <^=>- A is neither i-removable nor i-addable. 
Furthermore, i/sjA < A ; then SjA = AW, and i/s^A > A ; i/ien (sjA)w = A. 

Example 4.1. If A = (3, 1) then for type £>, C case we have the following examples: 



A = 





1 


2 


sqX = 





1 


2 


siA = 





1 


2 


S2A = 





1 




S3A = 





1 


2 


3 


























1 




















where 


we 


illed c(a) in 


each box a. For the 


same A in 


type D case, we have 


A = 


1 


2 


3 


sjA = 


1 


2 


3 


siA = 


1 


2 


3 


S2A = 


1 


2 


3 


s 3 X = 


1 


2 








1 








1 
















1 


2 






1 





4.4. Grassmannian elements in the finite rank case. Let SV(n) denote the set 
of strict partitions such that A C p n . Consider a subgroup of W(X n ) generated by 
si, . . . , s n _i, which is isomorphic to S n . If we denote it simply by S n , we have the fol- 
lowing natural bijection: 

SV(n) = W{C n )/S n = W(B n )/S n * W(D n+1 )/S n+1 . 
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Indeed, when we identify W(Coo) with S^, a set of coset representatives for W(C n ) / S n is 
given by W(C n ) nS^, which is naturally identified with SV{n). Similarly, W(D n+ i)/S n+ i 
can be identified with W(D n+1 ) n = SV{n). 

4.5. The ring 72 and W action. So far we have worked over the ring of coefficients 
Z 62, • • •]• Now we need to work over a larger coefficient ring: 

(4.8) 72 := Z[f3][b 1 ,b 2 ,...,b h b 2 ,...)/(b i ®b- i \ i > 1) 

This is essentially the ring of Laurent polynomials in infinite variables with coefficient in 
Z[/3]. One see that 72 is isomorphic to Z[/3][6 1; b 2 , . . .] [(1 + /36 i )- 1 (z > 1)] (see §£3D and also 
to Z[(3][bi,b 2 , . . .][(1 + /9^) _1 (z > 1)]. 72. has a natural action of the Weyl group defined 
by w(bi) = b w(i) , w(bi) = (w G W, i = 1, 2, . . .) with /3 fixed. 

4.6. Roots. We fix notation about root systems. Let L denote the free Z-module with 
basis {t;}i>i. The positive roots A + C L (set A - := — A + the negative roots) are defined 
by 



Type Boo ■ 


A+ 


= {U \ i> l}U{i,±^ | 


j > i > 1}, 


Type Coo : 


A+ 


= {2ti | i > 1} U {tj±ti 


\j>i>l} 


Type : 


A+ 


= {t j ±t i | j > 1 > 1}. 





For a positive integer n, define A+ := A + fl L n where L n := ^™ =1 Ztj. These are the 
corresponding positive root system of finite rank n of type B n ,C n , and D n . 
The following elements of A + are called the simple roots: 





Type Boo 


: a 


= £1, a» 


= ii+l 


-*i (i>l), 






Type Coo 


: a 


= 2*i, a 


= £j+ 


i-ti (i > 1), 






Type 7J>oo 


: a j 


= h + 1 2 , 


a« = 


t i+1 -U (z > 1). 




We can define 


a map e 


: L = 


0i=l 


-> 72 


satisfying e(ij) = 6j, e(- 


-k) = bi, 


a + 7) — e ( a ) 


©e( 7 ), e( 


a - 7) 


= e(a) G 


e(7) 


'a, 7 G L). Note that the 


map e is 



compatible with the natural action of W on L and the action on 72 defined in §4.51 For 
simple roots ccj, the explicit form of e(«j) written in terms of fej's are given by 

Boo ■ e(a ) = bi, e(a^) = b i+1 Qh(i> 1), 

Coo : e(a ) = 61 © 61, e(oi) = e fej (i > 1), 

TJoo : e(aj) = 61 © 6 2) e(a>i) = b i+l Qbi(i> 1). 

5. GKM RING AND ITS SCHUBERT BASIS 

In this section we introduce a ring which is defined by the i^-theoretic GKM condi- 
tion. A notion of "Schubert classes" is defined in a combinatorial way. 

5.1. GKM ring \P. Let Fun(5P,72) denote set of all maps from SV to 72. Fun(5P,72) 
is naturally an 72-algebra; the 72-module structure is given by diagonal multiplication, 
and multiplication is defined in point wise manner. For each a G A + we have a = w(cti) 
for some i G / and w EW. Then let s a = wSiW~ l . 

Definition 5.1 (GKM ring). Let \1/ be the subring ofFun(SV,TZ) defined by the following 
condition: 

ip(s a n) — ipifj) G e(a) ■ 72 for all \i G SV and a G A + . 
This condition and the associated geometry is discussed in §8.31 
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5.2. Divided difference operators on We now define the divided difference opera- 
tors iTi (i G /) on \l/ by the following formula: 

This can be rewritten as follows: 

5.2 (^iW)W = 7 ? • 

If if) G \l/ then the family {(vr'iV') (/•*)}> is actually gives an element of This fact can be 
shown by a similar argument in [TH] (the first Lemma in Appendix). 

The operators iii satisfy following relation: If W is type (Boo) then 7if = — f3iii (i = 
0,1,2...) and 

(5.3) iiiiXi+iiXi = 7r i+ i7Ti7ri + i (i > 1), 

(5.4) TTiTTj = TTjlTi (l, j > 1, \l - j \ > 2) , 

(5.5) 7r 7ri7r 7ri = 7ri7r 7ri7r , 7r 7ri = 7T;7r (i > 1). 
If W is type D M then = (i = 1, 1, 2 . . .), (jOj) . (IO> . and 

(5.6) TTi^i = ^271^2, 7Tj7Tj = VTj7Tj (j ^ 2). 

Remark 5.1. The operators 7Tj are 'left' divided difference operators. These operators are 
not appear explicitly in [20], where 'right' divided difference operators are used. Unfor- 
tunately we cannot find a literature that clarify a geometric origin of 'left' one. However, 
it turns out that the left divided difference operators are very useful especially for 'para- 
bolic' situations. See [H] and [19] for some discussions and applications of the left divided 
difference operators in equivariant cohomology. 

5.3. Schubert classes. 

Definition 5.2. A set of elements {ip\\ X G SV} C ^ is called a family of Schubert 
classes if the following conditions are satisfied: 



(5.7) 7iiip } 




if SiX < X 
if s iX > X 



(5.8) ^a(0) — <^a,0 (Kronecker's delta). 

A family of Schubert classes exists. This fact can be proved by a geometric argument 
in §BJ We have a proof of the existence as a consequence of Thm I7.ll 

Lemma 5.1. Let {^a}a be a family of Schubert classes. Suppose \i be a strict partition 
such that fi j£ 0. Let i G / be such that Sj/i < fi. Then 



(5.9) iPM 



(1 + f3e(-ai))si(ipx(siti)) + e(-ai)si(ip Si x(sifi)) if SiX < X 
Si (if)\(sin)) if SiX > X 



Proof. This is a direct consequence of the divided difference equation. □ 

Remark 5.2. Recurrence equation (15.91) is called 'left-hand' recurrence in [23], Remark 
2.3. 

Proposition 5.1. A family of Schubert classes, if exists, is unique. 
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Proof. Let {ip\}\esv be a family of Schubert classes. By definition, we have ip\{ty) = 
5\$. Suppose ix ^ 0. Then there is i G / such that Sifi < \i. By the recurrence equation 
f 15 .91) . \^\)\{p) | A G is uniquely determined from {ip\(sifi) | A G SV}. Hence by using 
induction on we conclude the uniqueness. □ 

Proposition 5.2. Let {^aIa be a family of Schubert classes. 

(1) If A (t fi then ip\(fi) = 0. 

(2) We have ip x (X) = ELeiMA) e (~ a )' where Inv(A) = {a G A + | s Q A < A}. 

Proof. (1) We prove by induction on For = 0, the vanishing property hold by the 
initial condition. Let fi ^ 0. There exists i such that Sifi < fi. Then we have A (f_ Sifi. So 
by inductive hypothesis we have ip\{sijj) = 0. Thus if s^A > A then from (15. 9p we have 
ipx(fx) = 0. Next suppose SjA < A. Then since both A and fi are i-removable, we have 
SjA G: By inductive hypothesis, we have ipx(sifi) = ip Si \(si[i) = 0. Hence from (15.91) 
we have = 0. 

(2) There exists i such that SjA < A. By the recurrence equation (15. 9p together with 
^\(si\) = as a consequence of (1), we have ^a(A) = e(—ai)si(ip Si \(si\)). By this 
equation ^>a(A) are determined inductively. We see that Ilaeinv(A) e (~ a ) satisfies this 
equation since we have Inv(A) = Sj Inv(sjA) U {c^}. □ 

Proposition 5.3. Let {^a}a be a family of Schubert classes. Any element in \l/ can be 
uniquely expressed as a possibly infinite IZ-linear combination of ip\ (A G SV). 

Proof. Let t/j G Define Supp(^) = {// G SV | ^(//) ^ 0}. Let A G Supp(^) be 
a minimum element in the ordering of strict partitions by inclusion. From the GKM 
condition and Prop. I5.2[ we see that ip(X) is divisible by e(— at) for all a G Inv(A). We see 
that the elements {e(— a) \ a G Inv(A)} are relatively prime, and hence ijj(\) is divisible by 
their product ^a(A) = rLeinv(A) e (~ a )- Let ip' = ip — ip(X) /ip\(\) ■ By the minimality 
of A in Supp(^) and Prop. 15.21 (1), Supp(?//) ^ Supp(-?/0- By repeating this, we may write 
ip as a possibly infinite 7£-linear combination of ipx's. □ 

6. Divided difference equation for GP\(x\b) and GQ\(x\b) 
In this section, we prove a divided difference equation for GP\(x\b) an d GQ\(x\b). 

6.1. Ring GY-ji and inverse limit of the A'-theoretic factorial P- and Q-functions. 

Let X be B, C or D. In order to state results for different types in a parallel way, we 
use the following convention: We denote by GY^ the following : GY^ = 1Z <8>z[/3] GT + , 
GY^ = GY^ = 1Z <S>z[/3] GY. We often suppress X when there is no fear of confusion. 
For each A G SV n , we define GX { " \x\b) by 



(6.1) 
(6.2) 



GB™(x\b) 
GCP(x\b) 

GDt\x\b) 



GPxixt, . . .,x n \0,b 1 ,b 2 , • • •), 
GQx(x u . . .,x n \b 1 ,b 2 , . . •), 

{GP\(x 1 , . . . ,x n \bi,b 2 , ■ ■ •) if n is even 
GP\(xi, . . . , x ni 0|&i, b 2 , . . .) if n is odd. 



(6.3) 



We can define GX x (x\b) = hmGXf \x\b) G GY 
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Proposition 6.1. Any f(x) G GT^ can be expressed uniquely as a possibly infinite 1Z- 
linear combination of GX\(x\b) 's, 

(6.4) f(x) = c a • GX x (x\b), c x G K, 

such that for all positive integer n the set {A G SV n \ c\ ^ 0} is finite. 

Proof. Using Lem. 12.51 in place of Cor. 12.11 we can show this by the same argument as 
in g321 (cf. Prop. EDO and Prop. [33]). □ 

6.2. Action of W on the K-supersymmetric algebras. We define an action of 
W{X OQ ) on GT£. Let (i > 1) act on K <& m GT by 

C a ® Q J =^ S i( C a) ® <Pa (C a £ K, <f) a E GT) , 
a / a 

where Si(c a ) is defined in §4.51 The action of Sq is defined as follows: For <fi G GT, define 

(6.5) (s o 0)(xi, x 2 , ■■■) = (p(b 1 ,x 1 , x 2 , ...), 
which is a well-defined element in TZ GT. In general, define 



(6.6) 



s ^2c a <j) a = ^2s {c a ) ■ S {<f) 

a) ifioi 

en, <p a e GT). 

\ a / a 



We have Sq = id on 71 ®z\p\ GT by virtue of the i^-supersymmetric property (cf. [§]). 
Define also sj = So s i s o- 

Proposition 6.2. The operators s, (i G /) give an action ofW(X OQ ) on GT^. 

Proof. The proof is straightforward (cf. [9], Prop. 7.2). □ 

6.3. Divided difference operators n,i on GT^. The divided difference operator -k^ on 
GT^ is defined by 

, fi7l F s,F-(l + /3e(a,))F ^ 
(6.7) 7TjF = — for all F G GTZ- 

e(aii) 

One can check directly that SiF — F is divisible by e(ai) so the right-hand side is an 
element in GT^. 

Remark 6.1. Using equation (1 + /3e(ct,;))(l + /3e(— = 1 we can rewrite (16. 7p as 

e(-a,) 

Theorem 6.1. VFe /iai>e 

GX X (i)(x\b) if Si\ < A 



^GXxixlb) 



-13 GX x (x\b) if s t \ > A. 



Proof. We first prove type C case. Let % > 1. We will work over finite n variables 
. . . , £„. Since the operator 7Tj acts only non-trivially on b{ and £>j+i, we only have to 
show 

'[[x|6]] A(0 if SjA < A 
-/3[[x|6]] A if Si A > A. 



^[[x\b]\' 
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If Si\ > A then one sees that [[x|6]] A is symmetric with respect to bi and b i+ i. It follows 
that 7iiGQ\(x\b) = —f3GQ\(x\b). If s^A < A then A& — i + 1 for some k. Then we note that 
rTj<fc[[ ;c l^]] Aj ' * s symmetric with respect to bi and fej+i, and nj>fc[M ]] Aj does not depend 
on bi and b i+ \. So we only have to calculate [[xfc|o]] Afc . It is easy to check the equation 

x k © b i+1 - (1 + (3b i+1 © bi)(x k © bi) = b i+1 © bi, 

which is equivalent to 7T;(xfc © bi) = 1. So we have 7r, = [[xfc|o]]\ 

Next we consider the operator ir . Define S^r = {w £ S n \ w{r + 1) < ■ • ■ < w(n)}. 
Then the definition of GQ\(xi, . . . , x n , x n+ i \ Q b\, o 2 , • ■ •) reads 
(6.8) 

r r n+1 

^ 'J^(i) ©x lu(i ))(x u ,(i) ©0i)(x™(i) © 6 2 ) • ■ ■ ©6Ai-i) x ] wt ™ J • 

t«eSn+i, r i=l i=l i=i+l ° 

Suppose so A < A then we have Xj > 2 for 1 < j < r — 1, A r = 1. We will prove 

GQ\(x!, . . .,x n ,bi\ © h,b 2 , ...)-(! + P{h © pi))GQaQei, • • .,x n \bi,b 2 , ■ ■ .) 

oiffi&i 

(6.9) = G(3 A (o)(ii, . . . ,x n |6i,6 2 , • • ■)■ 

If w(k) — n + 1 for some fc such that 1 < k < r — 1 then the corresponding term in (16.81) 
vanishes when we substitute x n+ i = b 1 . Define S^ +1 r to be the set of elements w G S n+ ± >r 
such that w(r) = n + 1. Then the corresponding part of (16. 8p . after substituting x n+ i = b±, 
becomes 

n+1 , r— 1 r— 1 / n+1 

c«o) m.) e nS"nM^n ..n ^M 

w€S£ +1 p J=r+1 w i=l «=1 \j=i+l, ]j=r y > w . 

where we used an obvious equation 

(6.11) (x w(t) © h) ( x ^® b A = x K 

\X w (i) © 01/ 

Define w' G S* n by = w(i) (1 < % < r — 1), = + 1) (r < i < n). Then 

to' G S n>r and this correspondence gives a bijection «S^ +1 r — > S n>r -i. Then (16 . 1 j) is written 



■ 



as follows: 



(6.12, M) y. n^-nii^if-n n :;;'^'!; 

w'eS n , r _i j=r Ul U Xw ^ i=l »=1 \j=i+l ™ W W X ™'C?) . 

Define S" +lr to be the set of elements w G S n+ x tr such that u>(n +1) = n + 1. We 
calculate the corresponding part in (16.81) . after substitution x n+ \ = b±. This, using (16.111) 
again, is equal to 

(6.13) Y Xw{r) ® \ l x f][[x w{l) \b}}^ x(f]f] ^ ® XwU) 

Let F\(x) denote the last function. The natural embedding S n C S n+ i given by w(n + 
1) = n + 1 gives a bijection = S^ +1 r . Using the identity 

(6.14) x ^l^l- {1 + m(Bbl)) - 



iCiu(r) © &1 ^tu(r) © 01 
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we have 

X\ i . . . , X n 1 0\ , O2 , • • .J 

it e 61 fi Vti iiii ^« e ^) 




= (61 ©60 ^ 

Ul&Sn^r K 1 \j=r + l v 7 

r— 1 /r— 1 n 

(6.15) xnMP . nn ^a|^a 

Now using (I6.12p . (I6.15p . and (12. 3p in Lemma [2. 1^ with m — n — r + 1, t = bi, we reduce 

(ED. 

Next suppose soA > A, which is equivalent to A r > 2, where r is the length of A. We 
show 

GQ x (x ly . . . ,x m b x \ 61,62, • • •) = GQ x (xi, . . . ,x n \bx,b 2 , . . .)■ 
In (16. 8ft . those w G S n+ i with 1 < w(n + 1) < r will vanish after the specialization x n+ i = 
b\. Using (16.111) . we see that the remaining terms, i.e. those elements satisfying w(n+l) = 
n + 1, coincide with the summands of GQ x (x±, . . . , x n \bi, b 2 , . . .) in the definition. 

Next we consider the case of type B. Calculations for equations for m (z > 1) are almost 
the same as for the case of type C. Indeed, we now use x Yli=\( x h) instead of [[x|6]] fc . 
For the equation with respect to the operator ttq, if soA < A, we use the identity 

(6.16) X ^® h ' - (1 + 0h) = bl(2 + ^ )) 

X w ( r ) © Ol Xw{r) © 61 

instead of (I6.14p . Noting that x w t r )(2 + f}x w ( r )) = x w m © x w ^ the rest is quite similar 
using the same equation (12. ip in the same way. The case of s A > A is the same as for 
type C. 

Finally we consider the case of type D. We also use finite n variables x±, . . . , x n , but 
note that n should be even here. Calculations for equations for 7Tj (i > 2) are quite 
similar to the case of type C. For the equation with respect to the operator 7Tj, we need 
to calculate the function 

(6.17) GP x (x 1 ,...,x n ,b 1 ,b 2 \ eb 1 ,eb 2 ,b 3 , ...). 

The case sjA > A is easy. Indeed (16.171) is equal to GP\(xi, . . . , x n \bi, 62, 63, . . .). For the 
case sjA < A, we can deduce 

GP x (x 1} ...,x n , 61, 6 2 | © 61, 06 2 , 63, ...)-(! + 0{pi © b 2 ))GP x (x 1 , . . . , x n \b u 6 2 , 63, . . j 

61 ©6 2 

= GP A(i) (xi, ... ,x n \h, 62,63, .. .) 
by using the following identities 

(6.18) £^_ (1 + Wie62 )) h ^ h - 



X w { r ) © 61 X w (r) © 61 

Ui © t "1 r Ui © Uj "1 r t ffi Itj 



(«») n^n^+n 

i=l j^ti J i=l 
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where the latter holds for even integer m. The last equation is exactly (12. 2p for k = and 
n odd. Equation for iri is □ 

7. Localization map 

In this section we define localization map $. This map gives an injective 7£-algebra ho- 
momorphism from the ring of i^-supersymmetric functions into \L The image of GX x (x\b)'s 
under the map $ is shown to be a (unique) family of Schubert classes. 

7.1. Vanishing property. Let v be the Grassmannian element in W corresponding to 
a strict partition /x, i.e. X(v) = /x (X + (v) = fi for type D case) in the notation of §4.31 
Define a sequence 6 M of elements in 1Z by 



b v (i) if v{i) is negative 
otherwise. 



(&,)« = 

Explicitly the sequences for types B, C £1X6 cLS follows: 



bp = (06 M1 ,..., 06^,0, 



where r is the length of /x, and for type D: 

36 M1+ i, . . . , e^+i, 0, . . .) if r is even 

. . . , eV+i- © & i ; 0, . . .) if r is odd. 



b, 



Remark 7.1. We defined 6 M for /x G P n in §2.41 (type A case). Notify that the above 
definition is for types B, C, and D. 

Proposition 7.1 (Vanishing property). Let A, /x be strict partitions. Then GX x (b^\b) = 
unless Xc fx and GX x (b x \b) = ELgi^a) e(-a). 

Proof. We only show this for GPx(x|5) and the case when the length r of /x is even. 
Other cases are similar. Using the stability property, GP x {b^\b) can be evaluated as 

GP x (eb fll+1 ,...,eb llr+1 \b). 

Suppose A <f_ /x. So we have fij < Xj for some 1 < j < r. Then we easily see that the 
polynomial rii=i[ a; '«(«)l^] Ai vanishes for any w G S r when we specialize Xi to Qb^ H+1 (1 < 
i < r). Note that the denominator of the factor 



i=1 j=i +1 ^(i) © x w(j) 



nn 



does not vanish identically under the specialization. Thus we have GPxib^b) = 0. 

Next we consider the case A = /x. We see that the terms other than the one comes from 
w = e are vanish, by the similar reason to the previous case. The term corresponding to 
w = e is evaluated by using the following equation: 

(e6„ +1<B fe)" 6*) n t b h x,+ 't b h w 

m 0a,+i o Xk+ i 



k=i+l 



II (eh i+ i®h) J] (eb Xt+1 eb Xk+1 ), 

fce{l,...,A i }\u£ =i+1 {A fe +l} k=i+l 

which is a simple consequence of cancellation. This last expression is readily seen to be 

n aG i„v(A) e (-«)- ° 
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7.2. Algebraic localization map <3>. Let \x be a strict partition. Define an 7£-algebra 
homomorphism $ M : GT-ji — > 1Z by "substitution" x = b ll . This is well-defined. In fact, we 
know that arbitrary element F(x) in GT^ can be written as a possibly infinite 7?.-linear 
combination F(x) = J2\esv c ^ ' GX\(x\b) of GX\s. Since GXxib^b) can be non-zero for 
only finitely many A's such that A C /i, -F(fc At ) = J2\esv c ^ ' GXxib^b) is a well-defined 
element in 7?.. 

Definition 7.1 (Localization map). Define the homomorphism of IZ- algebras 

^■.GVl^¥nn{SV,ll) f ^ (v ^ <&„(/)). 

$ is called the localization map /or GT^. 

We often call $ the algebraic localization map in order to form a contrast to geometric 
one. See $8] for geometric background of the map $. 

Proposition 7.2. Im($) C \&. 

This is a consequence of the following more explicit statement. 

Lemma 7.1. Let Uj = s tj - ti , % = s tj+ti (j > % > 1), s« = s u = s 2ti - If F(x) is a 
K- super symmetric function in GT then, for any ji G SV , we have 

(7.1) Fibt^-Fib^eibjeh) forj>i>l, 

(7.2) F(b 8i . ll )-F(b tl )e{b j (Bb i ) forj>i>l, 

(7.3) f(y-%)e(i.) for i > I. 
If F(x) is in GT + then we have 

(7.4) F(b Siifl ) — F(bfj) G (fej © fej) /oH > 1. 

Proof. There are the following possibilities: (i) z, j G A, (ii) i, j ^ A, z G A and j ^ A, 
(iv) z ^ A and j G A. Actions of t^- and are given as follows: 





i, j G A 


i,j i a 


% G A, j ^ A 


i ^ A, j G A 


tij ■ A 


A 


A 


(A\{<})U{j} 


(A \ {j}) U {*} 


Sij ■ A 


A\0,J'} 


AU{i,j} 


A 


A 



We may concentrate on two variables Xi,Xj. The check for ( 17. II) is easy in view of the 
fact that F is symmetric (note that (e(— a)) = (e(a))). To show that (17. 2p . it suffices to 
consider the cases (i) and (ii) (the cases (iii) and (iv) are obvious). Let G = F(Qbi, Qbj) — 
F(0,0). Then G as a polynomial of 6, vanishes at Qbj, because G is i^-supersymmetric. 
This implies that G is divisible by bj © b\ (cf. Lemma [3. 2p . Hence we have (17. 2p . 

Next we show (1731) and (17T4]) . It is obvious that F(e& 4 ) - F(0) is divisible by e&». The 
divisibility for F(Qbi) — F(0) by bi © 6j is the very condition that F is a member of GT + . 
This conclude the proof. □ 

The following is the main result of this paper. 
Theorem 7.1. {<&{GX\(x\b))}\(zsv is a family of Schubert classes. 

Proof. By the uniqueness of Schubert classes (Prop. 15. ip . it suffices to check the defining 
property. We know that <$>{GX\(x\b)) G * by Prop. OJ Note that $ o m = 7Tj o $ 
holds (cf. [9], Prop. 7.4.). By this fact and Thm. loTTj the divided difference equation 
(15. 7p is satisfied. The initial condition (15. 8p is satisfied because GX\(x\b) vanishes at 
x = b $ = (0, 0, . . .) for A ^ 0, and we have GX % (x\b) = 1. □ 
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Corollary 7.1. $ is injective. 

Remark 7.2. One can prove the injectivity of $ in the same way as [9], Lemma 6.5. 

8. Equivariant A"-theory of the maximal isotropic Grassmannians 

In this section, we show an application of the A-theoretic factorial P-and Q-functions 
to the equivariant A-theory of the maximal isotropic Grassmannians. 

8.1. Maximal isotropic Grassmannians. Let n be a positive integer. Suppose G is 
one of the groups SO{2n + 1, C), Sp(2n, C), or SO(2n + 2, C). Let T be a maximal torus 
of G, and let B be a Borel subgroup containing T. Then we have the corresponding root 
system, which we identify it with the one defined in l4.6l for types B n , C n , D n+ i respectively. 
Let P be the standard maximal parabolic subgroups corresponding to the simple root a 
for B n and C n , and a\ for D n+ \. Let Q n denote the homogeneous variety G/P. Then Q n 
is OG(n, 2n + 1), LG(n), OG{n + 1, 2n + 2) respectively for type B n , C n , and D n+ i, where 
LG(n) denote the Lagrangian Grassmannian, and OG(n, 2n + 1), OG(n + 1, 2n + 2) the 
maximal odd and even orthogonal Grassmannians. It is known that OG(n, 2n + 1) and 
OG(n + l,2n + 2) are isomorphic as algebraic varieties, however, note that here we are 
considering actions of different tori. 

We denote by the set of T-fixed points in Q n . We have a natural bijection = 
SV(n), where SV(n) is the set of strict partitions A such that A C p n . Let e\ G G% denote 
the T-fixed point corresponding to A G SV{n). Let £>_ be the opposite Borel subgroup, 
so that we have B D B- = T. The Schubert varieties Q\ in Q n is defined to be the closure 
of the orbit B_e\ C G/P. Then the co-dimension of Q\ is given by |A|. Note that e M G fix 
if and only of A C fi. 

Let Kt{Gti) denote the Grothendieck group of the abelian category of T-equivariant 
coherent sheaves on Q n . It is known that Kj<(G n ) has a natural ring structure. Since 
Q\ is T-stable, its structure sheaf Oq x defines a class [C^Jt in K T {Q n ). K T {Q n ) has a 
natural i?(T)-algebra structure, where R(T) is the representation ring of T, which is also 
identified with K^pt). As an i?(T)-module A^(^ n ) is free and has basis consisting of the 
Schubert structure sheaves [Cnjr (A G SV{n)). 

8.2. GKM ring. Let A(n) + denote the set of positive roots associated to the pair (G, B). 
Then A(n) + is for types B,C, and A^ +1 for type D (see §4.61 for the notation A+). 
We realize R{T) as the ring of Laurent polynomials : R{T) = 7L\e ±tx , . . . , e ±tn ] for types 
B, C and R(T) = Zfe** 1 , . . . , e ± *™+ 1 ] for type D. Let us denote by Fun(<SP(n), R(T)) the 
set of all functions from SV(n) to R(T). This is naturally an i?(T)-algebra by pointwise 
multiplication. 

Definition 8.1. Let ty n be the R{T)-subalgebra of Fun(SV(n), R(T)) defined as follows: 
a map ip : SV(n) — > R(T) is in ty n if and only if 

^{s a fi) - ip(fi) G (1 - e a )R{T) for all /i G &P(n), a G A(n) + . 

Letting (3 = — 1 we consider R(T) as a subalgebra of Z^z^Tl = Z[e ±t:L , e ±t2 , . . .}. If A, ji 

are strict partitions in SV(n), we see that G R(T). Thus we have ip^ 1 ' := ip\\sr{n) G 

Fun(SV(n), R(T)), which is obviously an element in \l/ n . Thus we have the following: 

Proposition 8.1. # n = ® XeSV(n) R(T)^ . 
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8.3. K T (G n ) and its Schubert basis. Note that K T (Qj t ) = Y[ e ,,&g T ^r( e **) is naturally 
identified with F\m(SV(n), R(T)), since we have Ql ^ SV (n . 

Theorem 8.1 ([20J). Let i be the inclusion map ^ Q n . The induced map 

i* : K T {Q n ) — > K T {Gl) = Fim(SP(n),R(T)) 

is injective, and its image is equal to the R(T)-subalgebra fy n . 

Proof. For the proof, see [20], Thm. 3.13 and Corollary 3.20. □ 

Theorem 8.2 ([20]). The image i*[O nx ] T , A G SP(n), ^s equal to ip { ™ ] G * n , where ^ 
is the restriction of to SV(n). 

An i?(T)-basis {t w } W £W for K~t{G/B) was constructed in [20]. The class t w is closely 
related to [CqJt but not-exactly coincides with it, where fl w is the Schubert variety in 
G/B corresponding to w G W. For the precise comparison with the class [Oh„]t ; see e.g. 
[23], [30] • Now we need the corresponding result for parabolic case, i.e. for K~t(G/P). 
The reader can consult [23] for the result. 

For X = B,C,D, we denote GT* = GF° = GT n and GT° = GT n>+ . We consider 
i?(T)-algebras R(T) <S>z\j3] GF* , with the specialization (3 = —1. For any strict partition 
A in SV n , let GX^ l \x\l — e*) be functions in R(T) ®^[/3] GF*, defined as follows: 

GB { "\x\l - e* 1 , . . . , 1 - e*», 0, . . .) if X = B, 

GQ^\x\l - e\ . . . , 1 - e*», 0, . . .) if X = C, 

GP^\x\l - e*\ . . . , 1 - e*», 1 - e*™+\ 0, . . .) MX = D. 

See gO for notation CXf \x\b). Then {GXf - e*) | A G <SP n } form an i?(T) basis 
of R{T) ® m GF*. 

Theorem 8.3. There exists a surjective homomorphism of R(T)- algebras 

ir n : R(T) ® m GF* — > K T (G n ), 

which sends GX^\x\l — e*) to [C^ a ]t A C p n and to if \ </i p n - 

Proof. The algebraic localization map $ naturally induces a homomorphism of R(T)- 
algebras $ n : i2(T) ® z[fl GT* -> given by = ($ M (F)) Me5P(n) . By Thm.Owe 

know that $ n maps CrX^(:r|l — e') to ^>j^ if A C p n and to if A <jL p n . Then there is a 
unique i?(T)-algebra homomorphism 7r n making the following diagram commutative: 

R(T) ® m GF* X T (£ n ) 

= 

Since i* is an isomorphism (Thm. IHTTj) . 7r n is determined by the above diagram, and has 
the desired property. □ 

Now we derive the non-equivariant analogue of Thm. lBTBl We first recall that two cases 
B n and D n+ i coincide in non-equivariant setting. 

Let e : R(T) — > Z be the ring homomorphism given by the evaluation at the identity 

element of T; it is given by e li \— > 1 (1 < % < r) or equivalently b\ i— > 0. Let GB ™\x) = 
GDf\x) = GP x (x x , ...,x n ) and GC^\x) = GQ { x n \x u . . .,x n ). 
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Corollary 8.1. There exists a surjective homomorphism of rings 

7r n : GF* — ► K(g n ), 

which sends GX^ (x) to [Oq x ] if A C p n and to if \ (jt p n - 
Proof. This follows from Thm. 18.31 since the canonical map 

(8.i) z®mT)K T (g n )^K(g n ), 

is an isomorphism of rings, where Z is considered as an i?(T)-algebra by e (see [20 
Proposition 3.25). □ 



9. Combinatorial expressions 

In this section we present two combinatorial expressions for GQ\(x\b) and GP\(x\b) 
(Thm. |9~T1 and Thm. [9~2|) . The proof will be given in 



9.1. Shifted set-valued tableaux. For each strict partition A, let D(A) be the shifted 
diagram of A, which is defined as D(A) = £ ZxZ | i < j < Xi+i — 1, 1 < % < 

Let A denote the ordered alphabet 

1' < 1 < 2' < 2 < • • • < n' < n. 

Let X denote the set of all non-empty subsets of A. 

A set-valued shifted tableau T of shape A is an assignment T : D(A) — > X such that 

(1) maxT(z, j) < minT(i,j + 1), maxT(z, j) < minT(i,j + 1), 

(2) Each a (= 1,2, ... ,n) appears at most once in each columns, 

(3) Each a' (= 1', 2', . . . , n') appears at most once in each rows. 

We denote by T(A) the set of all set-valued shifted tableaux of shape A. Define T'(A) 
to be the subset of T(A) consisting of T e T(A) satisfying the condition that for each 
i (1 < i < n), T(i, i) contains only 1, 2', 3, 4', . . . if i is odd, and V, 2, 3', 4, ... if i is even. 
Define T"(A) to be the subset of T(A) consisting of T G T(A) satisfying the condition 
that for each i (1 < i < n), T(i, i) contains no primed symbols {1', 2', . . . , n'}. 

Example 9.1. The following are examples of set- valued shifted tableaux of shape A = 
(4,3,1) : 



Ti 



1' 



12' 23 34' 



4' 



6' 



6 



1 


12' 


2 


2 




23' 


3 


3 






4' 





12' 23 34' 



4' 



6 



6 



Of these three tableaux in T(A), only T 2 is in T'(A), and only T 3 is in T"(A). 
For each integers i,j such that 1 < i < n, i < j and a G A, we define 



wi(i,j;a) 



x a ®b 
X\a\ © b 



if ae{l,2,.. 
if aG{l',2',. 



where bo = and 



w n {i,j;a) 



x a © if a G {1,2, . 

£| | 9 fy-i+i if ae{r,2', 



n} 

X}, 

. ,n} 
..,n'}. 
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For T 6 T(A) we define 

(9.1) (x\b)J = w T (i,j;a), (x\b) T n = w n (i,j;a). 

(tJ)eB(A), aeT(i,j) (iJ)6B(A), aeT(i,j) 

Recall that GX^\x\b) is denned in gO 

Theorem 9.1 (Combinatorial formula in terms of set- valued tableaux). Let X be a strict 
partition of length < n. Then we have 

(9.2) Type B : GB^\x\b) = £ ^\\x\b)] 

TeT"(X) 

(9.3) TypeC: GC^\x\b) = £ ^~^(x\b)J 

TeT(\) 

(9.4) TypeD: GD™(x\b) = ^ ^ X \x\b) T n 

TeT'(A) 

For example, let Ti be the tableau in Example 19.11 then the corresponding summand 
on the right-hand side of ( 19.31) equals: 

xi{xx © b 1 )x 2 (x 2 © bi)(x 2 © b 2 )(x 3 © b 2 )(x 3 © b 3 )(x A © &i)(x 4 © h)x e (x 6 © b 2 ). 

9.2. Excited Young diagrams. Let T> n denote the subset of Z x Z consisting of (i, j) 
satisfying 1 < z < n, % < j. We call an element in 2\ a 6ox Let D be a finite set of 
boxes, i.e. a finite subset oiV n . Suppose G D and j), (i, J + l) ^ -D 

(if i = j, the condition (i + 1, j) D is vacuous). Then it is said that D' — (D\ U 
(z + 1, j + 1) is obtained from D by an elementary excitation, and we denote D — > D' . 
Let S^(X) denote the set of all subsets D of V n obtained from D = O(A) by a sequence 
of successive elementary excitations 

D ^ £>!->• ► D r = D. 

Let D G £^(A). We denote by -B / (-D) the set of boxes satisfying the following 

conditions: 

(1) i D, 

(2) there is a positive integer k such that (z + k, j + fc) G D, 

(3) (z + s, j + s) ^ D for 1 < s < k, 

(4) (z + s, j + s + 1) ^ L> for < s < k, 

(5) (z + s — 1, j + s) ^ -D for < s < k (this condition is ignored if % = j). 
Example 9.2. Let n — 4. The following are example of elements of £{ (A) for A = (4, 2). 







X 










■ 




X 














X 






X 








X 






X 














X 




X 








X 









































The boxes with symbol x are the elements of B I {D). 

Next we consider the case of type D. In the sequel, we assume n is even, whenever 
we consider type D case. In order to treat type D case, we define S^(X), which is a 
subset of S^(X) consisting of EYD whose way of excitation of each diagonal box (z, z) is 
restricted to be even steps. More precisely, elementary excitation of (z, 1) G D is defined 
as follows: if (z, i) G D and (z, i + 1), (z + 1, i + 1), (z + 1, i + 2), (i + 2, z + 2) ^ D then 
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D' = (D \ U (i + 2,i + 2) is an excitation of type II. For off-diagonal boxes the 

definition of excitation is the same. For example the left diagram of Example 19.21 is in 
S^(X), but the right one is not. 

Let D G £^(X). We define the subset B U (D) of B J (D) by restricting k to be even in 
the definition of B^D). 

Example 9.3. In the following figure, the boxes with symbol x are the elements of B n (D). 







X 














X 






X 


















■ 























(9.5) 



wt B {i,j) 



Define for (i, j) G T> n , the following weight of types B, C, and D : 

if j ~ n , wt°(i,j) {'' '° 

For type D, note that we assumed n is even. We define 

%i ® Xj + i if j < n — 1 



if j < n 
if j > n 



wt D {t,j] 



Xi ® b 



'j-n+1 



if j > n 



Example 9.4. Let n = 4. The weight function on D 4 is given as follows: 



Xi©Xi 


2a £ 


Bx 2 


Xii 


Bx 3 


Xii 


Ba: 4 


x x i 


Bbi 


Xii 


Bb 2 


Xii 


Bb 3 






x 2 i 


Bx 2 


X 2 i 


Bx 3 


x 2 i 


Dx 4 


x 2 i 


Bh 


x 2 i 


Bb 2 


x 2 i 


Bb 3 










x 3 i 


Bx 3 


x 3 i 


Bx A 


x 3 i 


Bbi 


x 3 i 


Bb 2 


x 3 t 


Bb 3 














x A i 


Bx A 


x A t 


Bh 


x A i 


Bb 2 


x A i 


Bb 3 





while wt D is as follows: 



Xi®X 2 


X\®X 3 


X\ ®x A 


xi®h 


xi®b 2 


xi®b 3 


x±®b A 






x 2 ®x 3 


x 2 ®x A 


x 2 ®b 1 


x 2 ®b 2 


x 2 ®b 3 


x 2 ®b A 








x 3 ®x A 


x 3 ®bi 


x 3 ®b 2 


x 3 ®b 3 


x 3 ®b A 








x 4 ®bi 


x A ®b 2 


x A ®b 3 


x A ®b A 





We denote S*{\) = £%{\) = S%(\) and ££(\) = S*{\). We also denote B B (D) 
B C (D) = B(DY(X) and B D (D) = B(D) 11 . Define 

wt x {D) = \{ wt x { h j) \{ (1 + Pwt x (i',f)). 

(i,j)€D (i> ,j')£B x (D) 

Recall the definition Q) of GX^ n \x\b). 

Theorem 9.2. Let A be a strict partition of length < n. Then we have 
(9.6) GXf \x\b) = wt x {D) 
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Consider the right diagram D in Example 19.21 The corresponding summands of the 
right-hand side of (19. 6 1) are equal to the following: 

wt B (D) = x 1 {x 1 ®x 2 )(x 2 ®x 4 )(l + f3(x 1 ®x 3 ))(x 3 ®b 2 ){l + (3{x 2 ®b 1 )) 

x 4 {l + (3{x 2 © x 2 ))(l + P{x 3 © x 3 )){x 4 © h), 
wt c {D) = (xi©xi)(xiffix 2 )(x2©x 4 )(l + (3{x 1 ®x 3 )){x 3 ®b 2 ){l + (3{x 2 ®b 1 )) 

(x 4 © x 4 )(l + (3(x 2 © x 2 )){l + f3(x 3 © x 3 ))(x 4 © bi), 
wt D (D) = (xi © x 2 )(x 1 x 3 )(x 2 © &i)(l + © x 4 ))(x 3 © 6 3 )(1 + /3(x 2 b 2 )) 

(x 4 © 6i)(l + 0(x 2 © x 3 ))(x 4 © 6 2 )- 
Note that Bn(D) is given in Example 19.31 

Corollary 9.1. The coefficients ofGX { ^\x\b) as a polynomial in x\, . . . , x n are in N[/3, b%, b 2 

9.3. Type A case. There is an analogous formula for G\(x\, . . . , x n \b). Define T> A = 
{(i, j) G Z 2 | 1 < i < n, j > 0}. Define for G T>£, the following weight of type A : 

wt A (i,j) = Xi © bj. 

For a partition X e V n the set of excited Young diagrams S^(X) is defined in similar 
way (see [10]). For D e S A (A) the set B A (D) is defined in the same way as in §9.21 

Proposition 9.1 (EYD formula for Type A). Let A be a partition in V n . Then 

G X (x u ... jXn \b)= Yi n wt n^j) n (i+p™rt(i',f))- 

D£S£{X) (i,j)eD (i>,j')£BA(D) 

Proof of this proposition can be given in the same line as for Thm. 19.21 The arguments 
are left to the reader because these are quite similar and easier than types B, C, and D. 
Furthermore, we can show G\(xi, . . . ,x n \b) is equal to a combinatorial expression in terms 
of set- valued tableaux given by McNamara (cf. Remark \2.3\} . 

10. Proofs of Thm. EH and Thm. I9T21 

In this section we give proofs of Thm. 19.11 and Thm. 19.21 To make them more com- 
prehensible, we consider type C case. The modifications for other cases are left to the 
reader. 

10.1. Locally equivalent weight functions. Let S be a subset of T> n . By a weight 
function on S we mean a map w from S to a commutative ring R. Suppose a weight 
function w on T> n is given. For a strict partition A G SV n , we denote 

E\{w) = W ( D )> ™(D):=l[w(c) H (l + (3w(c')). 

Dt£ n (\) c£D c'G-B(D) 

Two weight functions w and w' on T> n are equivalent if the equality E\(w) = E\{w') holds 
for all A G SV n . Two weight functions Wi and w 2 on S are locally equivalent on S, if for 
any weight function w on T> n \ S, the following two weight functions w\/ W\ and w V w 2 
on D n are equivalent: 

, ... |u;(a) if a G V n \ S ,. „ 

(wVWi){a) = { \\ nX i = l,2. 

wAa) it a G o 
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Example 10.1. We will show that the following two weight functions on T> 2 are equiva- 
lent for arbitrary x, y, z, etc. In other words, the two weight functions on S are locally 
equivalent, where S is the set of the gray boxes. 



X 


a®t 


a 


y 




X 


a 


a®t 


y 












z 







t 


z 





Let w,w' denote the above weight functions. If A 
sums are 



then the corresponding EYD 



E\(w) = x(a®t)a + x(a®t)z(l + (3a) 
E x {w') = xa(a®t)+xaz(l + (3(a®t))+xt(l + (3a)z 

We can easily check the equality E\(w) = E\(w'). One sees that the only remaining A 
which we have to check is A = rj ] . Then we have the following two EYD sums which 
are equal: 

E x (w) = x(a © t), E x (w') = xa + xt(l + /3a). 
Next lemma is a generalization of this example. 
Lemma 10.1. The following weight functions on S are locally equivalent : 



a n (B t 


a n 






a 2 ®t 


a 2 


on ©t 


a\ 





a 





a n © t 






a-2 


a 2 ®t 


a\ 


ai ©t 





a © t 



where S is the subset ofT> n+ i consisting of two columns (i-th and (i + l)-th) with i > n + 1. 
Lemma 10.2. The following weight functions on S are locally equivalent : 



© t 


h-i 








6i © t 


bi 


c©tffit 


c®t 


ai©t 


a 2 ®t 


a 3 ®t 






c 


ai 


a-2 


a 3 





h-x 


h-! © t 








bi 


h®t 


c 


c®t 


ai 


a 2 


a 3 






c®t®t 


ai ®t 


a 2 ®t 


a 3 ®t 





where S is the subset ofT> n consisting of two columns and rows (i-th and (i + l)-th) with 
i < n. 
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Lemma 110.11 and Lemma 110.21 can be proved by elementary but quite tedious calcula- 
tions. An alternative proof using the Yang-Baxter equation is available (the details will 
appear in [27]). 

10.2. Outline of the proof of Thm [921 For A G SV n , define 

E x {x\,...,x n \b)= ^2 wt ( D )- 

D&£ n (X) 

Proposition 10.1. E\(x\, . . . ,x n \b) is a K -supersymmetric polynomial in x\, . . . , x n . 

Proof. Let i be 1 < i < n- 1. We apply Lemma l"10.2l to z-th and (i + l)-th columns and 
rows with t = XiQXi + \. It turns out that E\(x\, . . . , x n \b) is invariant under the exchange 
of Xi and Hence E\(x\, . . . , x n \b) is symmetric. 

Next we show the cancellation property of Def. 11.11 Let x\ = t, x 2 = Qt. Then weight 
function is given as follows: 





tet 


t®x 3 


t®X A 




Wb 2 


mb 3 






etet 


®t®x 3 




©t©6i 


et®b 2 


et®b 3 








X3®X 3 


X 3 ©X 4 


x 3 ®bi 


x 3 ®b 2 


x 3 ®b 3 










x 4 ©6i 


x 4 ©6 2 


x A ®b 3 





This is obviously equivalent to 









t®x 3 


t®x 4 


t©6i 


Wb 2 


t®b 3 









et®x 3 


©t©x 4 


©t©6i 


emb 2 


®t®b 3 








x 3 ®x 3 


x 3 ©x 4 


x 3 ®bi 


x 3 ®b 2 


x 3 ®b 3 








©X4 


£ 4 ©6l 


x 4 ®b 2 


x 4 ®b 3 





Then the sum of weights over V n reduces to the sum over the subset of V n consisting of 
the boxes G T> n such that % > 3. Since this subset is identified with D„_ 2 of type C n - 2 
weights with respect to x 3 , . . . , x n , and hence the sum is equal to E\(0, 0, x 3 , . . . , x n \b). 
This implies E\(xi, . . . , x n \b) is i^-supersymmetric. □ 

Proposition 10.2 (Stability). E\(xi, . . . , x n , 0\b) = E\{x\, . . . , x n \b). 

Proof. To make the stability property clearer, it is useful to arrange the weight in the 
following way by using symmetry. 



x 3 ®x 3 


x 3 ®x 2 


X 3 ®Xi 


x 3 ®h 


x 3 ®b 2 


x 3 ®b 3 


x 3 ®b 4 






x 2 ®x 2 


X 2 ®Xi 


x 2 ®b x 


x 2 ®b 2 


x 2 ®b 3 


x 2 ®b 4 








X\®X\ 


xi®bi 


xi®b 2 


xi®b 3 


x±®b 4 





Set x n+4 = 0. Then the weight at the box (1, 1) is zero. So if (1, 1) G D for D G £ n+ i(A), 
then wt(D) = 0. The set consisting of D G £ n+ i(A) such that (1, 1) ^ D is naturally in 
bijection with £ n (X). This bijection yields the equation. □ 

Let us denote by ^(^1^) the inverse limit hm E^(xi, . . . ,x n \b) (cf. Prop. 110. 2p . By 
Prop. 11.11 we have the following: 
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Lemma 10.3. E x {x\b) e GTg. 



In order to prove Thm. l9T2l we need the following lemma: 

E x{i) (x\b) ifsiX < X 
-f3E x {x\b) ifstX > X. 

With this lemma at hand, the proof for Thm. l£h"2l is completed as follows. 



Lemma 10.4. iTiE x (x\b) 



Proof of Thm. \9.^L Lemma [10.31 enable us to send E x (x\b) by $ into By the same 
argument of proof of Thm. 17.11 Lemma 110.41 implies that {&(E x (x\b))} is a family of 
Schubert classes. Since we know {$(GQ x (x\b))} is a family of Schubert classes (Thm. 17.11) . 
the injectivity of $ and the uniqueness of a family of Schubert classes implies E x (x\b) = 
GQ x (x\b). □ 



10.3. Proof of Lemma 110.41 Let i > 1. We calculate SiE x (x\b), which is equal to the 
weight sum over S n (X) of the weights obtained from the original one by replacing (n+i)-th 
and (n + % + l)-th columns with the following left diagram: 



xi ffi b i+1 


xi ffi h 




x 1 ffi bi 


xi ffi b i+ i 


x 2 © b i+ i 


x 2 ffi h 


— > 


x 2 ffi h 


x 2 ffi b i+ i 












X n ffi 


x n ffi bi 




x n ffi bi 


x n ffi 













k+i © k 



We consider each D £ S n (X) as a subset in D n+1 , on which the values of weight on (n + 1)- 
th row are all zero. Applying Lemma 110. 1\ we can change the weight with the right one 
without changing the weight sum in total. There is an additional box at the bottom of 
(i + n)-th column, whose weight is e(ai) = b i+ i bi. For example if n = 3, i — 1 then the 
two weight functions looks as follows: 



X 3 ®X 3 


x 3 ®x 2 


X 3 ®Xi 


x 3 ®b 2 


x 3 ®b x 


x 3 ®b 3 


x 3 ®b 4 






x 2 ®x 2 


X 2 @Xi 


x 2 ®b 2 


x 2 ®bi 


x 2 ®b 3 


X 2 ffi&4 








XiffiXi 


xi®b 2 


x x @bi 


xi@b 3 


Xiffi6 4 























x 3 ffi x 3 


x 3 ®x 2 


X 3 ®Xi 


x 3 ®h 


x 3 ®b 2 


x 3 ®b 3 


x 3 ®b A 






x 2 ®x 2 


X 2 ®Xi 


x 2 ®h 


x 2 ®b 2 


x 2 ®b 3 


x 2 ®b 4 








X\@X\ 


Xiffifei 


x x @b 2 


xx@b 3 


X\®b A 











b 2 Qh 











We denote the first one by Si(wt) (function obtained from wt by exchanging bi and 
and the second one by wt + i. Thus we have the following: 

Lemma 10.5. For i > 1 we have SiE x (xi, . . . , x n \b) = Eoe4 +1 (A) wt+i(D). 



34 



TAKESHI IKEDA AND HIROSHI NARUSE 



Proof of Lemma \10.4 for i > 1. Suppose A is i-removable. We denote by £+*(A) the 
set of D e £ n +i(A) such that wt +i (D) ^ 0. We have a map : £^ l (A) — > S n (\( l >) defined 
by removing the box of content % having the largest row index. One readily see that r, is 
surjective. Fix D £ £ n (\^). It suffices to show the following equation: 



(10.1; 



E 

De£+ l (\), r z (D)=D 



wt +i (D) - (l + Pe(oi)) 



E 



wt(D) = e(ai)wt(D ) 



£>e£„(A), n(D)=D Q 



Note that each term in the left-hand side of 110.11 is clearly a multiple of wt(D ). The 
differences comes from the contributions of the box to be removed by ty An obvious 
equation 



i-l 



1 — 1 i— 1 



(10.2) 



^ a; JJ(1 + p aj ) - (1 + /3a r ) ^ Oi Y[{1 + 0aj) = a r 

i=l j=l i=l j=l 



depicted below gives (110. ip . 

(VVV\)-\(VV\) = \ 

If A is not z-removeble, one easily sees that SiE\(x\, . . . ,x n \b) = E\(xi, . . . ,x n \b). Then 
we have 7iiE\(x\b) = —/3E\(x\b). □ 



In order to prove Lemma fl 0.41 for i = 0, we need the following Lemma. Let wt+o denote 
the weight function on D n+1 defined similarly as wt +i (i > 1). 

Lemma 10.6. E x (x 1 , . . . ,x n ,bi \ © 61,62,63, . . .) = Y.Dee n+1 (\) wt +0 (D). 

If n = 2 then E\(x\, X2, 61 1 61, b 2 , 63, . . .) is given by the following table. 



x 2 ®x 2 


X 2 ®X\ 


x 2 ®bi 


x 2 Qbi 


x 2 ®b 2 


x 2 ©6 3 


x 2 ©6 4 






Xi®X 2 


xi06i 


xxQbi 


X!©6 2 


X!©6 3 


£l©&4 








61 ©61 


heh 


61 ©6 2 


61 ©6 3 


6l ©64 





Note that 61 © b\ = 0. By applying Lemma HO. II to two columns indicated by *, we can 
deform this as follows: 



x 2 ®x 2 


X 2 ®Xi 


x 2 ®b x 


x 2 ©6 2 


£2 ©61 


x 2 ©63 


x 2 ®b A 






Xi®Xi 


Xi©6i 


xi©6 2 


x-iQbi 


2l©&3 


Xi©6 4 








61 ©61 








61 ©63 


61 ©64 




Repeat this process sufficiently many times we 


have 


x 2 ®x 2 


X 2 ®Xi 


x 2 ©6i 


x 2 ©6 2 


x 2 ©6 3 


x 2 ®b 4 


x 2 ©6 5 






Xi®X\ 


xi®bi 


xi©6 2 


xi©6 3 


xi©6 4 


xi©6 5 








6iffi6i 

















The proof of Lemma 110.41 for i = is completed. 



^-THEORETIC ANALOGUE OF FACTORIAL SCHUR P- AND Q- FUNCTIONS 



35 



10.4. Exited Young diagrams and Set-valued Tableaux. Let E\{x\b) and T\{x\b) 
be the functions in the right-hand of (19.61) and (19.31) respectively. In this section we prove 
the following: 

Proposition 10.3. E x (x\b) = T x (x\b). 

The main idea of proof is to use equivalence of weight functions so that we have "sepa- 
ration of variables" . For example, let n — 2, and consider the weight function wt defined 
by ( 19.51) . One readily see that the following weight function is equivalent to the original 
one. 



xi 

x\ x\ (B X2 xi® b\ x\ ®b 2 %i © ^3 %i © b\ 

x 2 

x 2 x 2 ffi h x 2 ® b 2 

This can be further deformed to the following equivalent one: 

11 

X\ X\ © 61 X\® X 2 X\ © 62 X\® 63 X\ © 64 

x 2 e&i 

x 2 

x 2 x 2 © bi x 2 ® b 2 

We repeat this process. For example, if n = 3, the following weight function is equivalent 
to the original one. 

xi 

Xl Xi ffi 61 xi ffi 62 X2 ffi x% Xi ffi 63 Xi ©64 £1 ffi 65 xi © &6 Xi ffi 67 x\ ® b% x-l ffi bg 
0x2062 

0x 2 ef>i0 

x 2 0000000 

X2 X2 ffi &1 X2 ffi 62 X2 ffi X3 X2 ffi 63 X2 ffi &4 X2 ffi &5 

x 3 © f> 2 

x 3 © bi 

x 3 

X3 X3 ffi bi X3 ffi &2 



If we only consider a strict partition A such that Ai < 3, we can use the following weight 
of "separated" form: 



xiOO 000 000 

xi xi ffi fei xi ffi 62 

x2 62 00 000 

0x 2 e&i00 000 

x 2 00 000 

x 2 x 2 ffi 61 x 2 ffi b 2 

x 3 e 62 

x 3 ffi b 1 

x 3 



x 3 x 3 ffi bi x 3 ffl b 2 



In general, we consider a weight function of the above separated form. Precisely, for pos- 
itive integer k, we define the weight function wt sep on V nk+n _ k+1 as follows: wt sep (i,j) = 
unless j — i < k and (i = 2 (mod k + 1) or j = 1 (mod k + 1)). On j th column with 
j = (k + l)s — k (1 < s < n), wt scp (i,j) = x s ( j — k + 1 < i < j). On i th row with 
i = (k + l)t - k+ 1 (1 < t < n), wt sep (i,j) = x t © b^ (i < j < i + k - 1). 
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Let A be a strict partition. Suppose k is large enough so that k > Ai, and let wt sep be 



the weight function defined the the preceding paragraph, 
excited Young diagrams D such that wt sep (D) ^ 0. Then 



Let £ sep (A) denote the set of 



Ex{x\b)= Yl wt *ep( D )- 



£>e£ aep (A) 

We denote by 7o(A) the set of ordinary (non set- valued) tableaux of shape A. 

Lemma 10.7. We have a bijection £ scp (A) — > 7o(A). 

Proof. We use a term horizontal (resp. vertical) line to call a rows (resp. columns) 
having nonzero weights in wt sep . We put label % to the horizontal line having weights 
Xi © bj_i, and f to the vertical line having weight Xj bj^. Let D £ £ sep (A). Since each 
box in D is on some line, horizontal or vertical, we associate its label. Then we write the 
label in the original position of the box in D(A). This gives a tableau of shape A. The 
fact that the resulting tableau is semistandard is readily seen from the configuration of 
the lines. Conversely, let T £ 7o(A). Any box a in D(A) has the right (excited) position, 
say a', on the line having label T(a). Semistandardness insures that the set {a'} is an 
excited Young diagram in £ scp (A). □ 



Example 10.2. Tableau T 

1' 



1 


2 


3 




3' 





corresponds to the following EYD: 




Here B's are the elements in D whereas CPs are the elements in B(D). 

Let T in T(A). The map D(A) — > A defined by H- max T(i,j) gives an element 
in 7o(A). Denote the resulting element by max(T). We denote (x\b)J in (19. ip simply by 
{x\b) T . 

Lemma 10.8. Let T £ 7o(A) be correspond to D £ £ sep (A) by the bijection of Lemma 



10.1 Then 



10.3) wt scp (D)= P m ~^{x\b) T . 

TeT(A), max(T)=T 
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If this lemma is true, the proof of Prop. 110.31 completes as follows: 

MA*) = E Wt **p( D ) 

r>e£ae P (A) 

= E ( E ^\Ab) 

ToeTo(A) \TeT(X), max(T)=T 

= (3 lTHX Ub) T = Ux\b), 

TeT(A) 

where the the second equality is the consequence of bijection in Lemma fl . 71 and equation 

(HQ3D. 

Example 10.3. In order to see that equation (110.31) holds, it is convenient to use weights 
Xji © bj_i on the vertical line instead of Xj This device makes clear that each term 
in the right-hand side of (110. 3j) corresponds naturally to a term in suitably expanded form 
of wt sep (D). If D is the EYD in example 110.21 then wt sep (D) is equal to the product of 
the weight in the following boxes: 



(1 + /3x!')xi 


(1 + p{xi © 6i))(l + P{x v bi))(x 2 © &i) 


(1 + (3(x 2 ®b 2 ))(x 3 ®b 2 ) 




x 3 > 





First note that the factor x\(x 2 © bi)(x 3 © 62)^3', when we ignore primes, is equal to 
(x\b) T °. Expanding the remaining factors of the form 1 + j3wt sep (a), which comes from 
B(D), the weight wt sep (D) can be expressed as a sum over tableaux 



al 


b2 


c3 




3' 





where a,b,c are subsets of A such that a C {1'}, b C {1,2'}, c C {2}. Note that 
these tableaux are exactly T's in T(A) such that max(T) = To. One sees that such T 
gives a summand /3' T '~' A '(x|6) T . For example, if a = {1'}, b = {2'}, c = then for the 
corresponding T, we have the term 

(5 2 ■ xi>xi(x 2 > © bi)(x 2 © b 1 )(x 3 © b 2 )x y . 

We then ignore 'primes' to have the right weight (3 2 (x\b) T . Thus we have (110. 3p . 

The argument in Example 110.31 can be generalized to the following: 

Proof of Lemma \10.8\ For each c = G D(A), let R(c;T ) denote the set of letters 
a in A such that joining a into T at c gives a semistandard set-valued tableau. Now 
suppose, for all c G D(A), subsets S c in R(c;T ) are given. Then by joining S c into T at 
c for all c G D(A), we have a tableau T G T(A) such that max(T) = To. Conversely, all 
such tableaux are obviously given in this way. This correspondence gives 

(10.4) (x\b) T ° e U^ Scl U w ^^= E /3 |THTo W, 

(Sc) ceD (A) ceD(A) ae5 c TeT(A), max(T)=T 

where 5" c runs for all the subsets in R(c;T ), and w(c;a) is defined in §9.11 Note that 
(x\b) T ° = YlceD w ^sep{ c ) ■ Finally as illustrated in Example 110.31 we have 

e n p iSci n w ^ °) = n ( x + /^ww). 

(5 c ) ceD (A) ceD(A) a€5 c c£B(D) 



) 
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This together with f li04j) leads to (fT03|) . □ 

References 

[1] S. Billey, V. Lakshmibai, Singular loci of Schubert varieties, Progr. Math. 182 (2000), Birkhauser, 
Boston. 

[2] A. S. Buch, A Littlewood-Richardson rule for the AT-theory of Grassmannians, Acta. Math. 189 

(2002) 37-78. 

[3] A. S. Buch, V Ravikumar, Picri rules for the A'-thcory of cominuscule Grassmannians, preprint, 

arXiv:1005.2605, to appear in J. Rcine Angew. Math. 
[4] E. Clifford, H. Thomas, A. Yong, AT-theoretic Schubert calculus for OG(2, 2n + 1) and jeu de taquin 

for shifted increasing tableaux, arXiv:1002.1664v2. 
[5] Z. Daugherty, A. Ram, R. Virk, Afhnc and degenerate affine BMW algebras: The center, 

larXiv: 1105.42071 

[6] M. Goresky, R Kottwitz, R. MacPherson, Equivariant cohomology, Koszul duality, and the localiza- 
tion theorem, Invent. Math. 131 (1998) 25-83. 

[7] P. N. Hoffman, J. F. Humphreys, Projective representations of the symmetric groups. Q-functions 
and shifted tableaux Oxford Mathematical Monographs. Oxford Science Publications. The Clarendon 
Press, Oxford University Press, New York, 1992. 

[8] T. Ikeda, Schubert classes in the equivariant cohomology of the Lagrangian Grassmannian, Adv. 
Math. 215 (2007), 1-23. 

[9] T. Ikeda, L. Mihalcea, H. Naruse, Double Schubert polynomials for the classical groups, Adv. Math. 
226 (2011) 840-886. 

[10] T. Ikeda, H. Naruse, Excited Young diagrams and equivariant Schubert calculus. Trans. Amer. Math. 
Soc. 361 (2009), no. 10, 5193-5221. 

[11] T. Ikeda, H. Naruse, Double Schubert polynomials of classical type and Excited Young diagrams, 
Kdkyuroku Bessatsu Bll (2009),. 

[12] T. Ikeda, H. Naruse, Y. Numata, Bumping algorithm for set-valued shifted tableaux (extended ab- 
stract for FPSAC 2011, Reykjavik), Discrete Math. Theor. Comput. Sci. (online) Proceeding volume 
for fpsac 2011, 527-538. 

[13] V. N. Ivanov, The dimension of skew shifted Young diagrams, and projective characters of the infinite 
symmetric group. (Russian) Zap. Nauchn. Scm. S.-Pcterburg. Otdel. Mat. Inst. Stcklov. (POMI) 240 
(1997), Tcor. Prcdst. Din. Sist. Komb. i Algoritm. Mctody. 2, 115-135, 292-293; translation in J. 
Math. Sci. (New York) 96 (1999), no. 5, 3517-3530. 

[14] V. N. Ivanov, Combinatorial formulas for factorial Schur Q-functions, J. Math. Sci. vol. 107, No. 5, 
2001, 4195-4211. 

[15] V. N. Ivanov, Interpolation analogues of Schur Q-functions. Zap. Nauchn. Sem. S.-Peterburg. Otdel. 
Mat. Inst. Steklov. (POMI) 307 (2004), Teor. Predst. Din. Sist. Komb. i Algoritm. Metody. 10, 
99-119, 281-282; translation in J. Math. Sci. (N. Y.) 131 (2005), no. 2, 5495-5507. 

[16] A. N. Kirillov, H. Naruse, Construction of double Grothcndicck polynomials of classical type using 
Id-Coxeter algebras, preprint. 

[17] D. Knuth, Overlapping pfaffians, Electronic Journal of combinatorics, 3 (2)1996, 1-13. 

[18] A. Knutson, A Schubert calculus recurrence from the noncomplex W-action on G/B, arXiv:0306304. 

[19] A. Knutson, T. Tao, Puzzles and (equivariant) cohomology of Grassmannians. Duke Math. J. 119 

(2003) , no. 2, 221-260. 

[20] B. Kostant, S. Kumar, T-equivariant if -theory of generalized flag varieties. J. Differential Geom. 32 
(1990), no. 2, 549-603. 

[21] V. Kreiman, Schubert classes in the if -theory and equivariant cohomology of the Lagrangian Grass- 
mannian, arXiv:0602245. 

[22] S. Kumar, Kac-Moody groups, their flag varieties and representation theory, Progress in Math. 204, 
Birkhauser, Boston, 2002. 

[23] T. Lam, A. Schilling, M. Shimozono, A'-thcory Schubert calculus of the afhne Grassmannian. Com- 
pos. Math. 146 (2010), no. 4, 811-852. 

[24] I. G. Macdonald, Symmetric functions and Hall polynomials, 2nd edition, Oxford Univ. Press, Oxford 
1995. 

[25] P. J. McNamara, Factorial Grothendieck polynomials, Elec. J. Combin. 13 (2006), #R71. 



^-THEORETIC ANALOGUE OF FACTORIAL SCHUR P- AND Q- FUNCTIONS 



39 



[26] A. Molev, B. E. A. Sagan, A Littlewood-Richardson rule for factorial Schur functions. 

Trans. Amer. Math. Soc. 351 (1999), no. 11, 4429-4443. 
[27] H. Naruse, Excited Young diagram and Yang-Baxter equation, in preparation. 

[28] J. J. C. Nimmo, Hall-Littlewood symmetric functions and the BKP equation. J. Phys. A 23 (1990), 
no. 5, 751-760. 

[29] P. Pagacz, Algcbro-geometric applications of Schur S- and Q-polynomials. Topics in invariant theory 

(Paris, 1989/1990), 130-191, Lecture Notes in Math., 1478, Springer, Berlin, 1991. 
[30] W. Graham, S. Kumar, On positivity in T-cquivariant X-theory of flag varieties. Int. Math. Res. 

Not. IMRN 2008, Art. ID rnn 093, 43 pp. 
[31] S. R. Ghorpade, K. N. Raghavan, Hilbert functions of points on Schubert varieties in the symplcctic 

Grassmannian. Trans. Amer. Math. Soc. 358 (2006), no. 12, 5401-5423. 
[32] K. N. Raghavan, K. S. Upadhyay, Hilbert functions of points on Schubert varieties in orthogonal 

Grassmannians. J. Algebraic Combin. 31 (2010), no. 3, 355-409. 
[33] I. Schur, Ubcr die Darstellung der symmetrischen und der altcrnicrcnden Gruppe durch gcbrochcnc 

lincare Substitutionen, J. Rcine Angew. Math. 139 (1911) 155-250. 
[34] J. Stembridgc, Shifted tableaux and the projective representations of symmetric groups, Adv. Math. 

74, (1989) 87-134. 

Department of Applied Mathematics, Okayama University of Science, Okayama 
700-0005, JAPAN 

email address: ikeOxmath . ous .ac.jp 

Graduate School of Education, Okayama University, Okayama 700-8530, JAPAN 
email address: rdcvl654@cc . okayama-u .ac.jp 



